Providing Persistent Objects in Distributed Systems Barbara Liskov Miguel Castro Liuba Shrira Atul Adya Laboratory for Computer Science Massachusetts Institute of Technology 545 Technology Square Cambridge MA 02139 liskov castro liuba adya lcs mit edu Abstract THOR is a persistent object store that provides a powerful programming model THOR ensures that persistent objects are accessed only by calling their methods and it supports atomic transactions The result is a system that allows applications to share objects safely across both space and time The paper describes how the THOR implementation is able to support this powerful model and yet achieve good performance even in a wide area large scale distributed environment It describes the techniques used in THOR to meet the challenge of providing good performance in spite of the need to manage very large numbers of very small objects In addition the paper puts the performance of THOR in perspective by showing that it substantially outperforms a system based on memory mapped files even though that system provides much less functionality than THOR 1 Introduction Persistent object stores provide a powerful programming model for modern applications Objects provide a simple and natural way to model complex data they are the preferred approach to implementing systems today A persistent object store decouples the object model from individual programs effectively providing a persistent heap that can be shared by many different programs objects in the heap survive and can be used by applications so long as they can be referenced Such a system can guarantee safe sharing ensuring that all programs that use a shared object use it in accordance with its type by calling its methods If in addition the system provides support for atomic transactions it can guarantee that concurrency and failures are handled properly Such a platform allows sharing of objects across both space and time Objects can be shared between applications running now and in the future Also objects can be used concurrently by applications running at the same time but at different locations different processors within a multiprocessor or different processors within a distributed environment A long standing challenge is how to implement this programming model so as to provide good performance A major impediment to good performance in persistent object stores is the need to cope with large numbers of very small objects Small objects This research was supported in part by DARPA contract DABT63 95 C 005 monitored by Army Fort Huachuca and in part by DARPA contract N00014 91 J 4136 monitored by the Office of Naval Research M Castro is supported by a PRAXIS XXI fellowship This paper appears in the prooceedings of ECOOP 99 Current address Department of Computer Science Brandeis University Waltham MA 02254 can lead to overhead at multiple levels in the system and affect the cost of main memory access cache management concurrency control and recovery and disk access This paper describes how this challenge is met in the THOR system THOR provides a persistent object store with type safe sharing and transactions Its implementation contains a number of novel techniques that together allow it to perform well even in the most difficult environment a very large scale wide area distributed system This paper pulls together the entire THOR implementation explaining how the whole system works THOR is implemented as a client server system in which servers provide persistent storage for objects and applications run at client machines on cached copies of persistent objects The paper describes the key implementation techniques invented for THOR the CLOCC concurrency control scheme which provides object level concurrency control while minimizing communication between clients and servers the MOB disk management architecture at servers which uses the disk efficiently in the presence of very small writes to individual objects and the HAC client caching scheme which provides the high hit rates of an object caching scheme with the low overheads of a page caching scheme In effect the THOR implementation takes advantage of small objects to achieve good performance thus turning a liability into a benefit The paper also presents the results of experiments that compare the performance of THOR to that of C OS C OS represents a well known alternative approach to persistence it uses memory mapped files to provide persistent storage for objects and a 64 bit architecture to allow addressing of very large address spaces This system does not however provide the functionality of THOR since it supports neither safe sharing nor atomic transactions The performance comparison between THOR and C OS is interesting because the latter approach is believed to deliver very high performance However our results show that THOR substantially outperforms C OS in the common cases when there are misses in the client cache or when objects are modified The rest of this paper is organized as follows We describe the THOR model in Section 2 Section 3 presents an overview of our implementation architecture Sections 4 5 and 6 describe the major components of the implementation Our performance experiments are described in Section 7 and conclusions are presented in Section 8 2 Thor THOR provides a universe of persistent objects Each object in the universe has a unique identity a state and a set of methods it also has a type that determines its methods and their signatures The universe is similar to the heap of a strongly typed language except that the existence of its objects is not linked to the running of particular programs The universe has a persistent root object All objects reachable from the root are persistent objects that are no longer accessible from the root are garbage collected Applications use THOR objects by starting a THOR session Within a session an application performs a sequence of transactions a new transaction is started each time the previous one completes A transaction consists of one or more calls to methods of THOR objects The application code ends a transaction by requesting a commit or abort A commit request may fail causing an abort if it succeeds THOR guarantees that the transaction is serialized with respect to all other transactions and that all its modifications to the persistent universe are recorded reliably If the transaction aborts it is guaranteed to have no effect and all its modifications are discarded THOR