The Effects of Memory Rich Environments on File System Microbenchmarks An I A Wang Computer Science Department UCLA awang lasr cs ucla edu Geoffrey Kuenning Computer Science Department Harvey Mudd College geoff cs hmc edu Peter Reiher Gerald J Popek Computer Science Department UCLA reiher popek lasr cs ucla edu Keywords L2 cache memory file system benchmark microbenchmark Abstract File system performance has been greatly influenced by disk caching mechanisms As the size of memory increases common workloads are more likely to run completely from memory and the effects of L2 caching and underlying hardware are becoming more visible This paper investigates performance anomalies observed when measuring and comparing the memory performance of various leading file systems We discovered that without considering the effects of L2 caching policy memory footprints of file systems states of L2 cache and memory page alignments existing microbenchmarks could produce numbers that are significantly misleading and could result in poor designs and improper conclusions about relative file system performance We recommend that the design of file system microbenchmarks for memory rich environments should carefully consider the initial states of memory and L2 cache subtle warm up effects the cache eviction policy interactions between workload size and the memory and cache sizes and the management granularity of memory and the L2 cache 1 INTRODUCTION The performance of modern file systems is heavily dependent on multi level caching and minor tuning of the caching mechanisms can have huge effects on the final results In traditional benchmarking environments where disk is the primary storage medium the major caching effects on performance come from cached disk blocks in memory However as the memory size increases more programs execute entirely from memory with no disk activity The amount of memory available has made caching of disk contents vastly more effective However one result of this change is that the effects of L2 caching are becoming increasingly visible in benchmarks and in real programs In particular microbenchmark numbers which guide many important file system design decisions are now highly susceptible to the subtleties of underlying hardware Gerald Popek is also associated with United Online Currently popular file system benchmarks are designed to measure the file system s disk performance Exercising the disk with a sizable working set is still the dominant practice An implicit assumption was that memory performance numbers could always be obtained from these benchmarks by simply shrinking the size of the working set below the size of the physical memory However we discovered that benchmarks also need to consider L2 caching effects to provide true insight into file system performance Through the process of examining performance anomalies encountered when comparing memory based and disk based file systems this paper demonstrates how the L2 caching policy memory page alignment the state of the L2 cache and the sizes of file system footprints can significantly affect file system microbenchmark numbers in non intuitive ways To the extent that file systems are designed or chosen for use based on the results of such benchmarks these effects can be misleading In some cases these effects cause a 20 or greater deviation in the key metrics produced by these benchmarks This paper Section 7 also makes a number of recommendations on how to design microbenchmarks to measure file system performance in the emerging memory rich computing environments To our best knowledge this study is the first investigation into the memory performance of systems using existing microbenchmarks that are designed to measure the disk performance of file systems 2 MICROBENCHMARK ANOMALIES REVEALED During the microbenchmarking phase of our research on the Conquest memory based file system 18 we encountered a number of performance anomalies that seemed inexplicable The performance study involved both memory based and disk based file systems The memory based file systems were ramfs by Transmeta 14 and Conquest and the diskbased file systems were ext2fs 2 reiserfs 8 and SGI XFS 16 Without examining the details of individual file systems intuitively memory based file systems should be able to outperform disk based file systems under all circumstances However the results were surprising The first two major anomalies were found using the Sprite LFS large file microbenchmark 12 This discovery was particularly unexpected because of its popularity for evaluating file systems 12 13 9 15 The Sprite LFS large file microbenchmark writes a large file sequentially with fsync reads from it sequentially writes a new large file randomly with fsync reads it randomly and finally reads it sequentially The final read phase was originally designed to measure an important case for a log structured file system The file size we used was 1 MB This file fit into main memory for all file systems tested The detailed experimental settings are described in Section 3 second 700 first 600 500 MB s 400 300 200 100 The measurements for Conquest reiserfs and SGI XFS are omitted because the ramfs and ext2fs numbers capture the relevant differences between disk based and memory based file systems Conquest specific questions such as why Conquest can outperform ramfs are covered in 18 Both ramfs and ext2fs follow the VFS interface and use the same generic read and write routines provided by the VFS However ramfs uses the caching data structures under VFS to store file system contents and metadata directly and those temporary data structures do not provide persistence of data after a system reboot Ramfs should approximate the practical achievable bound for memory performance of file systems Ext2fs on the other hand uses disk as the final storage destination for data and metadata Ext2fs is also one of the most widely used file systems in the UNIX world and it outperforms other disk based file systems on a wide variety of benchmarks 2 0 sw sr rw rr sr SGI XFS reiserfs ext2fs ramfs Conquest Figure 1 Sprite LFS large file microbenchmark for one 1 MB file The benchmark consists of five phases sequential write sequential read random write random read and sequential read The graph shows two major anomalies 1 For the first sequential read memory based file systems are slower than some of the disk based file systems 2 Memory is believed to provide a relatively uniform access
View Full Document
Unlocking...