DOC PREVIEW
CMU CS 15319 - lecture

This preview shows page 1-2-3-22-23-24-44-45-46 out of 46 pages.

Save
View full document
View full document
Premium Document
Do you want full access? Go Premium and unlock all 46 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 46 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 46 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 46 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 46 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 46 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 46 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 46 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 46 pages.
Access to all documents
Download any document
Ad free experience
Premium Document
Do you want full access? Go Premium and unlock all 46 pages.
Access to all documents
Download any document
Ad free experience

Unformatted text preview:

Distributed File Systems 15-319, spring 2010 12th Lecture, Feb 18th Lecture MotivationFilesFile SystemGreat! Now how do you Share Files?History of Sharing Computer FilesFile Sharing (1/7)File Sharing (2/7)File Sharing (3/7)File Sharing (4/7)File Sharing (5/7)File Sharing (6/7)File Sharing (7/7)Distributed File SystemSimple Example of a DFSDFS RequirementsNetwork File System (NFS)More on NFSNFS TradeoffsAndrew File System (AFS)AFS Details File System (GFS)Architecture of GFSGFS Architecture (Continued)Master and Chunk Server ResponsibilitiesThe Design TradeoffHow about Clients who need Data?Differences in the GFS APIReplication in GFSConsistency in GFSMutation in GFSHadoop Distributed File System (HDFS)HDFS DesignHDFS Architecture IHDFS Architecture IIHow to Talk to HDFSAnatomy of an HDFS File ReadData Locality and “Rack Awareness”Rack Awareness in HDFSAnatomy of an HDFS File WriteReplica PlacementReplica PlacementCoherency in HDFSCurrent Shortcomings of HDFSApplicationsReferencesCarnegie MellonSpring 2010 ©15-319 Introduction to Cloud ComputingIntroduction to Cloud ComputingMajd F. SakrDistributed File Systems 15‐319, spring 2010 12th Lecture, Feb 18thCarnegie MellonSpring 2010 ©15-319 Introduction to Cloud ComputingLecture Motivation Quick Refresher on Files and File Systems Understand the importance of File Systems in handling data Introduce Distributed File Systems Discuss HDFSCarnegie MellonSpring 2010 ©15-319 Introduction to Cloud ComputingFiles File in OS? Permanent Storage Sharing information since files can be created with one application and shared with many applications  Files have data and attributesCouloris,Dollimore and Kindberg Distributed Systems: Concepts & Design Edn. 4 , Pearson Education 2005File lengthCreation timestampRead timestampWrite timestampAttribute timestampReference countOwnerFile typeAccess control listFigure 2: File attribute record structureCarnegie MellonSpring 2010 ©15-319 Introduction to Cloud ComputingFile System The OS interface to disk storage Subsystem of the OS  Provides an abstraction to storage device and makes it easy to store, organize, name, share, protect and retrieve computer files  A typical layered module structure for the implementation of a Non‐DFS in a typical OS:Directory module: relates file names to file IDsFile module: relates file IDs to particular filesAccess control module: checks permission for operation requestedFile access module: reads or writes file data or attributesBlock module: accesses and allocates disk blocksDevice module: disk I/O and bufferingCouloris,Dollimore and Kindberg Distributed Systems: Concepts & Design Edn. 4 , Pearson Education 2005Carnegie MellonSpring 2010 ©15-319 Introduction to Cloud ComputingGreat! Now how do you Share Files? 1980s: SneakernetCopy files onto floppy disks, physically carry it to another computer and copy it again. We still do it today with Flash Disks! Networks emerged Started using FTP Save time of physical movement of storage devices. Two problems:– Needed to copy files twice: from source computer onto a server, and from the server onto the destination computer. – Users had to know the physical addresses of all computers involved in the file sharing.Carnegie MellonSpring 2010 ©15-319 Introduction to Cloud ComputingHistory of Sharing Computer Files Networks emerged (contd.) Computer companies tried to solve the problems with FTP, new systems with new features were developed. Not as a replacement for the older file systems but represented an additional layer between the disk, FS and user processes. Example:Sun Microsystem's Network File System (NFS).Carnegie MellonSpring 2010 ©15-319 Introduction to Cloud ComputingFile Sharing (1/7) On a single processor, when a write is followed by a read, the read data is the accurate written one On a distributed system with caching, the read data might not be the most up to date. http://www.nmc.teiher.gr/activities/MASTERS/JOINT/Material/Vall/DSC_2.pdfCarnegie MellonSpring 2010 ©15-319 Introduction to Cloud ComputingFile Sharing (2/7) How to deal with shared files on a distributed system with caches? There are 4 ways!Carnegie MellonSpring 2010 ©15-319 Introduction to Cloud ComputingFile Sharing (3/7) UNIX semantics Every file operation is instantly visible to all users. So, any read following a write returns the correct value. A total global order is enforced on all file operations to return the most recent value. In a single physical machines, a shared l‐Node is used to achieve this control. Files data is a shared data structure among all users. In Distributed file server, same behavior needs to be done! Instant update cause performance implications. Fine grain operations increase overhead.Carnegie MellonSpring 2010 ©15-319 Introduction to Cloud ComputingFile Sharing (4/7) UNIX semantics Distributed UNIX semantics Could use centralized server that can serialize all file operations. Poor performance under many use patterns. Performance constraints require that the clients cache file blocks, but the system must keep the cached blocks consistent to maintain UNIX semantics. Writes invalidate cached blocks. Read operations on local copies “after” the write according to a global clock happened “before” the write.– Serializable operations in transaction systems.– Global virtual clock orders on all writes, not reads.Carnegie MellonSpring 2010 ©15-319 Introduction to Cloud ComputingFile Sharing (5/7) Session semantics Changes become visible when the session is finished. When modified by multiple parties, the final file state is determined by who closes last. When two processes modify the same file, session semantics would produce one process’ changes or the other but not both. Many processes keep files open for long periods.  This approach is different from most of programmers experience, so must be used with caution.  Good for process whose file modification is transaction oriented (connect, modify, disconnect).  Bad for series of open operations.Carnegie MellonSpring 2010 ©15-319 Introduction to Cloud ComputingFile Sharing (6/7) Immutable Files No updates are possible. Both file sharing and replication are simplified. No way to open a file for writing or appending. Only directory entries


View Full Document

CMU CS 15319 - lecture

Download lecture
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view lecture and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view lecture 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?