6 824 Spring 2006 6 824 Lab 5 Cache Consistency Due Lecture 12 Introduction Now it s time to add caching to ccfs in order to improve performance First you ll add a local write back block cache to each ccfs server Then you ll make the caches consistent by forcing write back of the dirty cached blocks associated with a filehandle and deletion of the clean blocks when you release the lock on that filehandle Finally you ll add lazy release i e caching of locks so that each ccfs can serve NFS RPCs without contacting the block or lock servers when possible Your server will be a success if it manages to operate out of its local block and lock cache when reading writing files and directories that other hosts aren t looking at but maintains correctness when the same files and directories are concurrently read and updated on multiple hosts Getting Started cd mkdir lab 5 rsync av lab 4 lab 5 cd lab 5 wget nc http pdos csail mit edu 6 824 labs test lab 5 c cc o test lab 5 test lab 5 c Testing Performance Our measure of performance is the number of puts and gets that your server sends to the block server You can tell the block server to print out a line every 100 puts gets telling you the current totals blockdbd v 35983 30 70 3200 6900 You may need to comment out debugging statements in the block and lock servers in order to see these print statements You should change the definition of DBG from 1 to 0 in lock server C Each line indicates the number of puts and the number of gets so far The workload on which we ll evaluate your server s performance is generated by test lab 5 It creates two Cite as Robert Morris course materials for 6 824 Distributed Computer Systems Engineering Spring 2006 MIT OpenCourseWare http ocw mit edu Massachusetts Institute of Technology Downloaded on DD Month YYYY sub directories and creates deletes 100 files in each directory using each directory through only one of the two servers ccfs dir1 localhost 35983 root file handle 4d1def4be444163c ccfs dir2 localhost 35983 4d1def4be444163c test lab 5 classfs dir1 classfs dir2 Create delete in separate directories OK Now look at the last pair of numbers printed by blockdbd Your code from lab 4 will probably generate a few thousand puts and perhaps twice that many gets Your goal is to reduce those numbers to about a dozen puts and at most a few hundred gets Of course your server must also remain correct so we will require the server you hand in to pass the Lab 4 testers as well as getting good performance on the Lab 5 tester Step One Block Cache In Step One you ll add caching to your block client blockdbc C and blockdbc h without cache consistency This cache will make your server fast but incorrect get should check if the block is cached and if so return the cached copy Otherwise get should fetch the block from the block server put it in the local cache and then return it to the file server put should just replace the cached copy and not send it to the block server You ll find it helpful for the next section if you keep track of which cached blocks have been modified by put i e are dirty remove should delete the block from the local cache Look in blockdb h and blockdb C for example code that stores blocks in an ihash Warning ihash insert adds a key value pair to the hash table without checking whether that key is already present so you should check if the key is there and remove it before inserting a new value You may find that your fs C code does not work correctly if get and put call the callback immediately as they would if they execute the operation in the cache without consulting the block server You can delay the callback void blockdbc get xcb callback void bool str ref cb str v cb true v void blockdbc get str key callback void bool str ref cb if key is in the cache str value delaycb 0 wrap this blockdbc get xcb cb value Cite as Robert Morris course materials for 6 824 Distributed Computer Systems Engineering Spring 2006 MIT OpenCourseWare http ocw mit edu Massachusetts Institute of Technology Downloaded on DD Month YYYY When you re done run test lab 5 giving the same directory twice and watch the numbers printed by the block server You should see zero puts and somewhere between zero and a few hundred gets or perhaps no numbers at all since blockdbd only prints a line for every 100 puts gets Your server should pass test lab 4 a pl and test lab 4 b if you give it the same directory twice but it will probably fail test lab 4 b with two different directories because it has no cache consistency Step Two Cache Consistency In Step Two you ll ensure that each get sees the latest put even when the get and put are from different ccfs servers You ll arrange this by ensuring that your server writes a file s modified dirty cached blocks back to the block server before the server releases the lock on that file Similarly your server should delete unmodified clean blocks from its cache when it releases the lock on the relevant file You will need to add a method to the block client to let your file server ask for a block to be eliminated from the cache This flush method should first check whether the block is dirty in the cache and send it to the block server if it is dirty Blocks that your server has removed with the block client s remove method should also be removed from the block server if the block server knows about them The flush method should take a callback argument that it calls when it is done so that your server can learn when the flush has finished Your server will need to call flush just before releasing a lock back to the lock server You could just add flush calls to fs C before each release However you ll find that Step Three is easier if you instead arrange for the lock client to call a callback in fs C when the lock client is about to send a RELEASE RPC to the lock server A clean interface might involve fs fs registering a callback with the lock client which the lock client calls with a lock name whenever it about to send a RELEASE RPC So you could add this code to fs C fs fs blockdbc xdb lock client xlc db xdb this lc xlc this lc set releasing cb wrap this fs do release ADD And add code like this to lock client h class lock client public void set releasing cb callback void str callback void void ref ptr cb releasing cb cb private callback void str callback void void …