Goals for Today CS162 Operating Systems and Systems Programming Lecture 20 File Caching Durability Authorization Distributed Systems Reliability and Access Control Distributed Systems April 6 2010 Ion Stoica http inst eecs berkeley edu cs162 Note Some slides and or pictures in the following are adapted from slides 2005 Silberschatz Galvin and Gagne Gagne Many slides generated from lecture notes by Kubiatowicz 4 6 10 Directory Structure Hard link jim book count inode count Many systems allow directory structure to be organized as an acyclic graph or even a potentially cyclic graph Hard Links different names for the same file Pro Reference counting i e when all links are deleted the file is deleted Pro Efficiency one name resolution step to get inode Cons Restricted to same file system Multiple directory entries point at the same file Soft Links shortcut pointers to other files Soft symbolic link jim book avi avi book Implemented by storing the logical name of actual file Pro Can cross file systems Cons Inefficient multiple name resolution steps Cons No reference counting dangling pointers e g files directory is deleted Name Resolution The process of converting a logical name into a physical resource like a file Traverse succession of directories until reach target file Global file system May be spread across the network CS162 UCB Fall 2009 Lec 20 2 Directory Structure Not really a hierarchy 4 6 10 CS162 UCB Fall 2009 4 6 10 Lec 20 3 Page 1 CS162 UCB Fall 2009 Lec 20 4 Directory Structure Con t Where are inodes stored How many disk accesses to resolve my book count In early UNIX and DOS Windows FAT file system headers stored in special array in outermost cylinders Read in file header for root fixed spot on disk Read in first data block for root Table of file name index pairs Search linearly ok since directories typically very small Read Read Read Read Read in in in in in Header not stored near the data blocks To read a small file seek to get header seek back to data Fixed size set when disk is formatted At formatting time a fixed number of inodes were created They were each given a unique number called an inumber file header for my first data block for my search for book file header for book first data block for book search for count file header for count Current working directory Per address space pointer to a directory inode used for resolving file names Allows user to specify relative filename instead of absolute path say CWD my book can resolve count 4 6 10 CS162 UCB Fall 2009 4 6 10 Lec 20 5 Where are inodes stored CS162 UCB Fall 2009 Lec 20 6 Pros Later versions of UNIX moved the header information to be closer to the data blocks UNIX BSD 4 2 puts a portion of the file header array on each cylinder For small directories can fit all data file headers etc in same cylinder no seeks Often inode for file stored in same cylinder group Cylinder group one or more consecutive cylinders Superblock information about cylinder group size number of blocks etc Cylinder group map records the block usage within the cylinder File headers much smaller than whole block a few hundred bytes so multiple headers fetched from disk at same time Reliability whatever happens to the disk you can find many of the files even if directories disconnected Part of the Fast File System FFS General optimization to avoid seeks 4 6 10 CS162 UCB Fall 2009 4 6 10 Lec 20 7 Page 2 CS162 UCB Fall 2009 Lec 20 8 Linux Example Ext2 3 Disk Layout In Memory File System Structures Disk divided into block groups like cylinder groups Provides locality Each group has two block sized bitmaps free blocks inodes Block sizes settable at format time 1K 2K 4K 8K Open system call Resolves file name finds file control block inode Makes entries in per process and system wide tables Returns index called file handle in open file table Actual Inode structure similar to 4 2BSD with 12 direct pointers Read write system calls Ext3 Ext2 w Journaling Use file handle to locate inode Perform appropriate reads or writes Example create a file1 dat Several degrees of under dir in Ext3 protection with more or 4 6 10 Lec 20 9 CS162 UCB Fall 2009 less cost 4 6 10 File System Caching Key Idea Exploit locality by caching data in memory Cache Size How much memory should the OS allocate to the buffer cache vs virtual memory Too much memory to the file system cache won t be able to run many applications at once Too little memory to file system cache many applications may run slowly disk caching not effective Solution adjust boundary dynamically so that the disk access rates for paging and file access are balanced Buffer Cache Memory used to cache kernel resources including disk blocks and name translations Can contain dirty blocks blocks not yet on disk Replacement policy LRU Can afford overhead of timestamps for each disk block Advantages Read Ahead Prefetching fetch sequential blocks early Works very well for name translation Works well in general as long as memory is big enough to accommodate a host s working set of files Key Idea exploit fact that most common file access is sequential by prefetching subsequent disk blocks ahead of current read request if they are not already in memory Elevator algorithm can efficiently interleave groups of prefetches from concurrent applications How much to prefetch Disadvantages Fails when some application scans through file system thereby flushing the cache with data used only once Example find exec grep foo Other Replacement Policies Too many imposes delays on requests by other applications Too few causes many seeks and rotational delays among concurrent file requests Some systems allow applications to request other policies Example Use Once File system can discard blocks as soon as they are used CS162 UCB Fall 2009 Lec 20 10 File System Caching con t Name translations Mapping from paths inodes Disk blocks Mapping from block address disk content 4 6 10 CS162 UCB Fall 2009 4 6 10 Lec 20 11 Page 3 CS162 UCB Fall 2009 Lec 20 12 File System Caching con t Delayed Writes Writes to files not immediately sent out to disk Administrivia 3rd project due Monday April 12 Instead write copies data from user space buffer to kernel buffer in cache I ll be away next Wednesday Friday Eurosys Enabled by presence of buffer cache can leave written file blocks in cache for a while If some other application tries to read data before written to disk file system will read from cache Lecture will be taught by Ben Hindman No office hour on
View Full Document
Unlocking...