Review Example of Multilevel Indexed Files Multilevel Indexed Files from UNIX 4 1 BSD CS162 Operating Systems and Systems Programming Lecture 20 Key idea efficient for small files but still allow big files File Header format First 10 ptrs to data blocks Block 11 points to indirect block containing 256 blocks Block 12 points to doublyindirect block containing 256 indirect blocks for total of 64K blocks Block 13 points to a triply indirect block 16M blocks Reliability and Access Control Distributed Systems November 10 2008 Prof John Kubiatowicz http inst eecs berkeley edu cs162 UNIX 4 1 Pros and cons Pros Simple more or less Files can easily expand up to a point Small files particularly cheap and easy Cons Lots of seeks Very large files must read many indirect block four I Os per block 11 10 08 Review UNIX BSD 4 2 Inode Structure Same as BSD 4 1 same file header and triply indirect blocks except incorporated ideas from DEMOS Uses bitmap allocation in place of freelist Attempt to allocate files contiguously 10 reserved disk space Skip sector positioning Kubiatowicz CS162 UCB Fall 2008 Lec 20 2 Goals for Today File Caching Durability Authorization Distributed Systems BSD 4 2 Fast File System FFS File Allocation and placement policies Put each new file at front of different range of blocks To expand a file you first try successive blocks in bitmap then choose new range of blocks Inode for file stored in same cylinder group as parent directory of the file Store files from same directory near each other Note I put up the original FFS paper as reading for last lecture and on Handouts page Later file systems Clustering of files used together automatic defrag of files a number of additional optimizations 11 10 08 Kubiatowicz CS162 UCB Fall 2008 Lec 20 3 Note Some slides and or pictures in the following are adapted from slides 2005 Silberschatz Galvin and Gagne Gagne Many slides generated from my lecture notes by Kubiatowicz 11 10 08 Kubiatowicz CS162 UCB Fall 2008 Lec 20 4 Where are inodes stored Where are inodes stored Later versions of UNIX moved the header information to be closer to the data blocks In early UNIX and DOS Windows FAT file system headers stored in special array in outermost cylinders Often inode for file stored in same cylinder group as parent directory of the file makes an ls of that directory run fast Pros Header not stored near the data blocks To read a small file seek to get header seek back to data Fixed size set when disk is formatted At formatting time a fixed number of inodes were created They were each given a unique number called an inumber UNIX BSD 4 2 puts a portion of the file header array on each cylinder For small directories can fit all data file headers etc in same cylinder no seeks File headers much smaller than whole block a few hundred bytes so multiple headers fetched from disk at same time Reliability whatever happens to the disk you can find many of the files even if directories disconnected Part of the Fast File System FFS General optimization to avoid seeks 11 10 08 Kubiatowicz CS162 UCB Fall 2008 Lec 20 5 11 10 08 Linux Example Ext2 3 Disk Layout Kubiatowicz CS162 UCB Fall 2008 Lec 20 6 In Memory File System Structures Open system call Resolves file name finds file control block inode Makes entries in per process and system wide tables Returns index called file handle in open file table Read write system calls Example create a file1 dat under dir in Ext3 11 10 08 Kubiatowicz CS162 UCB Fall 2008 Lec 20 7 Use file handle to locate inode Perform appropriate reads or writes 11 10 08 Kubiatowicz CS162 UCB Fall 2008 Lec 20 8 File System Caching Key Idea Exploit locality by caching data in memory Name translations Mapping from paths inodes Disk blocks Mapping from block address disk content Buffer Cache Memory used to cache kernel resources including disk blocks and name translations Can contain dirty blocks blocks not yet on disk Replacement policy LRU Can afford overhead of timestamps for each disk block Advantages Works very well for name translation Works well in general as long as memory is big enough to accommodate a host s working set of files Disadvantages Fails when some application scans through file system thereby flushing the cache with data used only once Example find exec grep foo Other Replacement Policies File System Caching con t Cache Size How much memory should the OS allocate to the buffer cache vs virtual memory Too much memory to the file system cache won t be able to run many applications at once Too little memory to file system cache many applications may run slowly disk caching not effective Solution adjust boundary dynamically so that the disk access rates for paging and file access are balanced Read Ahead Prefetching fetch sequential blocks early Key Idea exploit fact that most common file access is sequential by prefetching subsequent disk blocks ahead of current read request if they are not already in memory Elevator algorithm can efficiently interleave groups of prefetches from concurrent applications How much to prefetch Too many imposes delays on requests by other applications Too few causes many seeks and rotational delays among concurrent file requests Some systems allow applications to request other policies Example Use Once 11 10 08 File system can discard blocks as soon as they are used Kubiatowicz CS162 UCB Fall 2008 Lec 20 9 File System Caching con t Delayed Writes Writes to files not immediately sent out to disk Instead write copies data from user space buffer to kernel buffer in cache Kubiatowicz CS162 UCB Fall 2008 Tuesday Tomorrow is a holiday However we will be having voluntary sections Interesting information there ext3 filesystem queuing theory if didn t get it last week Try to make it if you can MIDTERM II Wednesday December 3rd Disk scheduler can efficiently order lots of requests Disk allocation algorithm can be run with correct size value for a file Some files need never get written to disk e g temporary scratch files written tmp often don t exist for 30 sec Final Exam Disadvantages What if system crashes before file has been written out Worse yet what if system crashes before a directory file has been written out lose pointer to inode Kubiatowicz CS162 UCB Fall 2008 Lec 20 11 Lec 20 10 Administrivia Enabled by presence of buffer cache can leave written file blocks in cache for a while If some other application tries to read data before written to disk file system will read from cache Flushed to disk
View Full Document
Unlocking...