MORRIS A Distributed File System for Read Intensive Applications David Chau Jennifer Lin Michael Matczynski Nathan Palmer ddcc jwlin mikem palmer mit edu May 12 2005 1 Abstract Introduction There are a wide variety of data mining problems where applications make frequent file reads but infrequent writes spending as much as 98 99 of their time performing read related operations and as little as 0 05 writing to disk 11 Many of these data mining tasks particularly those in the field of bioinformatics lend themselves to a high degree of parallelism where multiple processes working individually on independent machines perform reads from the same underlying database and merge their results quickly when each node completes its portion of the work 11 5 12 Many of these data mining applications operate by sequentially reading large flat file databases processing each record in turn 1 5 12 The large size of these databases often on the order of several gigabytes 3 2 makes aggressive client side caching an impractical solution to improving performance The files are often simply too large to store wholesale in volatile memory on each client and even if the memory were available a node rarely re reads the same portion of a file meaning that the data would seldom be revisited after the initial read which caused the data to be cached Moreover this solution fails to address the issue of contention between clients attempting the initial caching read Because of the high degree of parallelism exploited by these applications providing concurrent read access to the shared data becomes an important factor in application performance Multiple nodes requesting the same file from one networked storage device can lead to clients sit This paper presents the design and implementation of Modularly Optimized Round robin ReadIntensive Storage MORRIS a file system which provides high throughput for read intensive applications NFSStripe MORRIS primary component is an NFS loopback server that achieves performance competitive with the traditional single server model by distributing the task of data storage and retrieval over multiple machines There are two main challenges associated with such a design The first consists of structuring the underlying storage of the filesystem in such a way as to take advantage of multiple data servers thereby allowing multiple concurrent read operations to be efficiently executed We solve this problem by striping files across multiple StripeServer data servers in fixed size blocks Employing multiple machines to serve data from disk allows our system to fulfill multiple client requests at once whereas a singleserver arrangement cannot The second challenge is to ensure filesystem coherence as multiple concurrent client operations are issued to multiple independent servers We solve this problem while conferring minimal impact on the system s performance by designing a multiple reader single writer locking protocol specifically suited to our system s data structures 1 between these two components to enable high read throughput while simultaneously ensuring the coherence of the filesystem That is because the clients are expected to make frequent concurrent read operations but infrequent writes we allow multiple NFSStripe servers to simultaneously read a file but require that operations which modify the filesystem be given exclusive access to those structures being modified The protocol that enforces this combined with a simple striping scheme where the blocks constituting a file are laid out over multiple StripeServer machines enables multiple clients to read the same file albeit different blocks simultaneously without suffering from delays caused by lock contention or overloaded data servers We would like our system to provide semantics to the client that mimic as closely as possible those of a single local disk since that is what most users and programmers have come to expect Put another way users and applications should notice no difference between interacting with a filesystem residing on a local disk versus one mounted via an NFSStripe server In this regard the choice of NFS as an interface to the client is a reasonable one since that was one of the original goals laid out by Sandberg et al 9 We note also that at a more technical level because its interface complies to the NFS v3 7 standard existing programs may access an NFSStripe filesystem without modification 7 Per client Throughput Mb sec 6 5 4 Series1 3 2 1 0 1 2 3 4 Number of Clients Figure 1 Performance of a single FreeBSD NFS server serving multiple clients We varied the number of clients concurrently reading a 200 MB file and observed a nearly linear decrease in per client throughput ting idle wasting valuable computing time while waiting for their data to arrive Considering the long running nature of these applications and the fact that they spend nearly all of their time reading we suggest that a file server capable of improving read access times for these applications could provide a significant boost in application performance This paper presents an NFS file server that provides high throughput for these readintensive applications Our goal is to alleviate the bottleneck that occurs when a single NFS server is used as the storage medium in such environments Figure 1 illustrates the performance degradation experienced by multiple clients reading the same file from a single NFS server Our system consists of two separate components NFSStripe which is an NFS loopback server and StripeServer a data block server that NFSStripe communicates with NFSStripe is intended to be run on each client requiring access to a common filesystem It reads and writes data in fixed size blocks by communicating with StripeServer data storage servers which maintain the blocks on stable media Observations about the workload characteristics of the applications that we expect our system to service in particular scientific data mining have allowed us to fine tune the interaction 2 Related Work In an effort to provide better performance scalability than the single server setup achieves we take an approach similar to that of Frangipani 10 by layering NFSStripe on top of multiple StripeServer data servers Similar to that of the Frangipani s sister service Petal StripeServer behaves like a network disk in that data may be read or written in blocks In contrast with Frangipani however NFSStripe server is a full featured NFS v3 7 loopback server 6 meaning