CS162Operating Systems andSystems ProgrammingLecture 20Reliability and Access Control /Distributed SystemsNovember 10, 2008Prof. John Kubiatowiczhttp://inst.eecs.berkeley.edu/~cs162Lec 20.211/10/08Kubiatowicz CS162 ©UCB Fall 2008Review: Example of Multilevel Indexed Files• Multilevel Indexed Files: (from UNIX 4.1 BSD)– Key idea: efficient for small files, but still allow big files– File Header format:» First 10 ptrs to data blocks» Block 11 points to “indirect block” containing 256 blocks» Block 12 points to “doubly-indirect block” containing 256 indirect blocks for total of 64K blocks» Block 13 points to a triply indirect block (16M blocks)• UNIX 4.1 Pros and cons– Pros: Simple (more or less)Files can easily expand (up to a point)Small files particularly cheap and easy– Cons: Lots of seeksVery large files must read many indirect block (four I/Os per block!)Lec 20.311/10/08Kubiatowicz CS162 ©UCB Fall 2008Review: UNIX BSD 4.2• Inode Structure Same as BSD 4.1 (same file header and triply indirect blocks), except incorporated ideas from DEMOS:– Uses bitmap allocation in place of freelist– Attempt to allocate files contiguously– 10% reserved disk space– Skip-sector positioning • BSD 4.2 Fast File System (FFS)– File Allocation and placement policies » Put each new file at front of different range of blocks» To expand a file, you first try successive blocks in bitmap, then choose new range of blocks– Inode for file stored in same “cylinder group” as parent directory of the file– Store files from same directory near each other– Note: I put up the original FFS paper as reading for last lecture (and on Handouts page).• Later file systems – Clustering of files used together, automatic defrag of files, a number of additional optimizationsLec 20.411/10/08Kubiatowicz CS162 ©UCB Fall 2008Goals for Today• File Caching• Durability• Authorization• Distributed SystemsNote: Some slides and/or pictures in the following areadapted from slides ©2005 Silberschatz, Galvin, and Gagne Note: Some slides and/or pictures in the following areadapted from slides ©2005 Silberschatz, Galvin, and Gagne. Many slides generated from my lecture notes by Kubiatowicz.Lec 20.511/10/08Kubiatowicz CS162 ©UCB Fall 2008Where are inodes stored?• In early UNIX and DOS/Windows’ FAT file system, headers stored in special array in outermost cylinders– Header not stored near the data blocks. To read a small file, seek to get header, seek back to data.– Fixed size, set when disk is formatted. At formatting time, a fixed number of inodes were created (They were each given a unique number, called an “inumber”)Lec 20.611/10/08Kubiatowicz CS162 ©UCB Fall 2008Where are inodes stored?• Later versions of UNIX moved the header information to be closer to the data blocks– Often, inode for file stored in same “cylinder group” as parent directory of the file (makes an lsof that directory run fast).– Pros: » UNIX BSD 4.2 puts a portion of the file header array on each cylinder. For small directories, can fit all data, file headers, etc in same cylinder⇒noseeks!» File headers much smaller than whole block (a few hundred bytes), so multiple headers fetched from disk at same time» Reliability: whatever happens to the disk, you can find many of the files (even if directories disconnected)– Part of the Fast File System (FFS)» General optimization to avoid seeksLec 20.711/10/08Kubiatowicz CS162 ©UCB Fall 2008Linux Example: Ext2/3 Disk Layout• Example: create a file1.datunder /dir/ in Ext3Lec 20.811/10/08Kubiatowicz CS162 ©UCB Fall 2008• Open system call:– Resolves file name, finds file control block (inode)– Makes entries in per-process and system-wide tables– Returns index (called “file handle”) in open-file table• Read/write system calls:– Use file handle to locate inode– Perform appropriate reads or writes In-Memory File System StructuresLec 20.911/10/08Kubiatowicz CS162 ©UCB Fall 2008File System Caching• Key Idea: Exploit locality by caching data in memory– Name translations: Mapping from paths→inodes– Disk blocks: Mapping from block address→disk content• Buffer Cache: Memory used to cache kernel resources, including disk blocks and name translations– Can contain “dirty” blocks (blocks not yet on disk)• Replacement policy? LRU– Can afford overhead of timestamps for each disk block– Advantages:» Works very well for name translation» Works well in general as long as memory is big enough to accommodate a host’s working set of files.– Disadvantages:» Fails when some application scans through file system, thereby flushing the cache with data used only once» Example: find . –exec grep foo {} \;• Other Replacement Policies?– Some systems allow applications to request other policies– Example, ‘Use Once’:» File system can discard blocks as soon as they are usedLec 20.1011/10/08Kubiatowicz CS162 ©UCB Fall 2008File System Caching (con’t)• Cache Size: How much memory should the OS allocate to the buffer cache vs virtual memory?– Too much memory to the file system cache ⇒ won’t be able to run many applications at once– Too little memory to file system cache ⇒ many applications may run slowly (disk caching not effective)– Solution: adjust boundary dynamically so that the disk access rates for paging and file access are balanced• Read Ahead Prefetching: fetch sequential blocks early– Key Idea: exploit fact that most common file access is sequential by prefetching subsequent disk blocks ahead of current read request (if they are not already in memory)– Elevator algorithm can efficiently interleave groups of prefetches from concurrent applications– How much to prefetch?» Too many imposes delays on requests by other applications» Too few causes many seeks (and rotational delays) among concurrent file requestsLec 20.1111/10/08Kubiatowicz CS162 ©UCB Fall 2008File System Caching (con’t)• Delayed Writes: Writes to files not immediately sent out to disk– Instead, write() copies data from user space buffer to kernel buffer (in cache)» Enabled by presence of buffer cache: can leave written file blocks in cache for a while» If some other application tries to read data before written to disk, file system will read from cache – Flushed to disk periodically (e.g. in UNIX, every 30 sec)– Advantages: » Disk scheduler can efficiently order lots of requests» Disk
View Full Document
Unlocking...