Retrospection on a Database System MICHAEL STONEBRAKER University of California at Berkeley This paper describes the implementation history of the INGRES database system. It focuses on mistakes that were made in progress rather than on eventual corrections. Some attention is also given to the role of structured design in a database system implementation and to the problem of supporting nontrivial users. Lastly, miscellaneous impressions of UNIX, the PDP-11, and data models are given. Key Words and Phrases: relational databases, nonprocedural languages, recovery, concurrency, protection, integrity CR Categories: 3.50, 3.70, 4.22, 4.33, 4.34 1. INTRODUCTION This paper was written in response to several requests to know what really happened in the INGRES database management system project [22] and why. To the extent that it contains practical wisdom for other implementation projects, it serves its purpose. To the extent that it is a self-righteous defense of the existing design, the author apologizes in advance. It may be premature to write such a document, since INGRES has only been fully operational for three years and user experience is still somewhat limited. Hence the ultimate jury, real users, has not yet made a full report. The reason for reporting now is that we have reached a turning point. Until late 1978, the goal was to make INGRES “really work,” i.e., efficiently, reliably, and without sur- prises (bugs) for users. There are now only marginal returns to pursuing that goal. Consequently, the project is taking new directions, which are discussed below. This paper is organized as follows. In Section 2 we trace the history of the project through its various phases and highlight the more significant events that took place. Then, in Section 3, we discuss several lessons that we had to learn the hard way. Section 4 takes a critical look at the current design of INGRES and Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the ACM copyright notice and the title of the publication and ita date appear, and notice is given that copying is by permission of the Association for Computing Machinery. To copy otherwise, or to republish, requires a fee and/or specific permission. The INGRES project was sponsored by the U.S. Air Force Office of Scientific Research under Grant 783596, the U.S. Army Research Office under Grant DAAG-29-G-0245, the Naval Electronics Systems Command under Contract NOOO39-78-G-0013, and the National Science Foundation under Grant MCS75-03839-ADl. Author’s address: Department of Electrical Engineering and Computer Science, University of Cali- fornia, Berkeley, CA 94720. @ 1980 ACM 0362-5915/80/0600-0225 $00.75 ACM Transactions on Database Systems, Vol. 5, No. 2, June 1980, Pages 225-240.226 * Michael Stonebraker discusses some of the mistakes. Next, Section 5 consists of an assortment of random comments. Lastly, Section 6 outlines the future plans of the project. 2. HISTORY The project can be roughly decomposed into three periods: (1) the early times- March 1973-June 1974; (2) the first implementation-June 1974-September 1975; (3) making it really work-September 1975-present. We discuss each period in turn. 2.1 The Early Times The project began in 1973 when Eugene Wong and I agreed to read and discuss literature relating to relational databases. From the beginning we were both enthusiastic about an implementation. It did not faze either one of us that we possessed no experience whatsoever in leading a nontrivial implementation effort. In fact, neither of us had ever written a sizable computer program. Our first task was to find a suitable machine environment for an implementa- tion. It quickly become clear that no machine to which we had access was appropriate for an interactive database system. Through various mechanisms (mainly engineered by Eugene Wong and Pravin Varaiya) we obtained about $90,000 for hardware. The liability that we obtained was a commitment to write a geodata system for the Urban Economics Group led by Pravin Varaiya and Roland Artle. Our major concerns in selecting hardware were in obtaining large (50 or 100 megabytes at the time) disks and a decent software environment. After studying the UNIX [18], I was convinced that we should use UNIX and buy whatever hardware we could afford to make it run. We placed a hardware order in February of 1974 and had a system in September of the same year. We decided to offer a seminar running from September 1973 to June 1974 in which a design would be pursued. Somewhat symbiotically the seminar split into two groups: One group, led by Gene, would plan the user language; the other group, led by me, would plan the support system. The language group converged quickly on the retrieval portion of the data sublanguage QUEL. It was loosely based on DSL/Alpha [5] but had no notion of quantifiers. As soon as UNIX was chosen, my group laid out the system catalogs (data dictionary) and the access method interface. Initially, we considered a nonrela- tional structure for the catalogs, as that would make them somewhat more efficient. However it quickly became clear that providing a specialized access facility for the system catalogs involved code duplication and would ruin the possibility of using QUEL to query the system catalogs. The latter feature would, in essence, provide a data dictionary system for free. Hence the system catalogs became simply more relational data for the system to manage. An idea from the very start had been to have several implementations of the access method interface. Each would have the same calling conventions for simplicity and