Foundation for Object / Relational Databases: The Third Manifesto

PREFACE: \ The Third Manifesto is a detailed proposal for the future direction of data and database management systems (DBMSs). Like Codd's original papers on the relational model, it can be seen as an abstract blueprint for the design of a DBMS and the language interface to such a DBMS. In particular, it lays the foundation for what we believe is the logically correct approach to integrating relational and object technologies—a topic of considerable interest at the present time, given the recent appearance in the marketplace of several "object/relational" DBMS products (sometimes called universal servers). Perhaps we should add immediately that we do not regard the idea of integrating relational and object technologies as "just another fad," soon to be replaced by some other briefly fashionable idea. On the contrary, we think that object/relational systems are in everyone's future—a fact that makes it even more important to get the logical foundation right, of course, while we still have time to do so.\ The first version of the Manifesto was published informally in early 1994 (though we had been thinking about the idea of such a document for several years prior to that time), and the first "official" version appeared in 1995. Since then we have presented the material in a variety of forms and forums and discussed it with numerous people—indeed, we continue to do so to this day—and we have refined and expanded the original document many, many times. We would like to stress, however, that those refinements and expansions have always been exactly that; nobody has ever shown us that we were completely on the wrong track, and developmentof the Manifesto has always proceeded in an evolutionary, not a revolutionary, manner. Now we feel it is time to make the material available in some more permanent form; hence the present book.\ One reason we feel the time is ripe for wider dissemination of our ideas is as follows. As already indicated, we see a parallel between the Manifesto and Codd's original papers on the relational model; like those papers of Codd's, the Manifesto offers a foundation for what (we believe) the database systems of the future ought to look like. Also like those papers of Codd's, however, the Manifesto itself is, deliberately, fairly terse and not all that easy to read or understand. Would it not have been nice to have had a book that documented and explained and justified Codd's ideas, back at the beginning of the relational era? Well, here we are at the beginning of "the object/relational era," and—modesty aside—we believe this book can play a role analogous to that of that hypothetical relational book. To that end, we have been careful to include not only the formal specifications of the Manifesto itself (of course), but also a great deal of supporting and explanatory material and numerous detailed examples.\ By the way, we should make it clear that our ideas do rest very firmly in the relational tradition. Indeed, we would like our Manifesto to be seen, in large part, as a definitive statement of just what the relational model itself consists of at the time of writing (for it too has undergone a certain amount of evolution over the years). Despite our remarks in the previous paragraph concerning "the object/relational era," therefore, the ideas expressed in the Manifesto must not be thought of as superseding those of the relational model, nor do they do so; rather, they use those ideas as a foundation and build on them. We believe strongly that the relational model is still highly relevant to database theory and practice and will remain so for the foreseeable future. Thus, we regard our Manifesto as being very much in the spirit of Codd's original work and continuing along the path he originally laid down. To repeat, we are talking evolution, not revolution.\ There is another point to be made here, too. Given the current interest in object/relational systems, we can expect to see a flurry of books on such systems over the next few years. However, it is unlikely, if history is anything to go by, that those books will concern themselves very much with general principles or underlying theory; it is much more probable that they will be product-oriented, if not actually product-specific. The present book, by contrast, definitely is concerned with theoretical foundations rather than products; in other words, it allows you to gain a solid understanding of the underlying technology per se, thereby enabling you among other things to approach the task of evaluating commercial products from a position of conceptual strength.\ While we are on the subject of commercial products, incidentally, we should make it clear that we ourselves have no particular commercial ax to grind. We regard ourselves as independent so far as the marketplace is concerned, and we are not trying to sell any particular product. The ax we do have to grind is that of logical correctness!—we want to do our best to ensure that the industry goes down the right path, not the wrong one.\ And in that connection, we would like to mention another reason we feel the book is timely: namely, the fact that the SQL standards bodies, both national and international, are currently at work on a proposal called SQL3 that addresses some of the same issues as our Manifesto does. An appendix to the present book gives a detailed set of comparisons between our ideas and those of the current SQL3 proposal.\ Note: Another body, the Object Database Management Group (ODMG), has also published a set of proposals that, again, address some of the same issues. Another appendix to this book therefore takes a look at the ODMG ideas as well.\ Two more special features of the book are the following:\ \ We define a new and simplified relational algebra, called A, which emphasizes rather more than previous algebras have done its firm foundation in predicate logic.\ \ \ We also define a database programming language called Tutorial D, which realizes the ideas of the Manifesto in concrete form and is used as the basis for examples and illustrations throughout the book.\ \ \ \ Finally, we should mention one further feature that we believe to be highly significant, and that is our proposal for a model of subtyping and inheritance. Many authorities have rightly observed that there is currently no consensus on any such model, and we offer our proposal for consideration in the light of this observation. Indeed, we believe we have some original—and, we also believe, logically sound and correct—thoughts to offer on this important subject. Part IV of the book (five chapters) is devoted to this topic.\ Structure of the Book\ The body of the book is divided into four major parts:\ \ Preliminaries\ Formal Specifications\ Informal Discussions and Explanations\ Subtyping and Type Inheritance\ \ Part I sets the scene by explaining in general terms what the Manifesto is all about and why we wrote it. It also contains an informal overview of two approaches to building an object/relational system, one of which is (we claim) right and the other wrong. We recommend that you read both of these chapters fairly carefully before moving on to later parts of the book.\ Part II is the most formal part. It consists of three chapters:\ \ Chapter 3 is the Manifesto itself—a "no frills" version, with virtually nil by way of illustration or further explanation. (For the benefit of anyone who might have seen earlier drafts of the Manifesto, we should explain that this "no frills" version consists essentially of just the formal text from those earlier drafts, with all commentary and suchlike material deleted.)\ Chapter 4 contains the definition, mentioned above, of our new relational algebra ("A").\ Chapter 5 defines the language Tutorial D. This language, which is (of course) based on the principles laid down in the Manifesto and on our new algebra A, serves as the basis for examples throughout the remainder of the book. It also serves to suggest what an implementation of the ideas of the Manifesto might look like in actual practice.\ \ Note: Most of the material of these three chapters is provided primarily for purposes of reference; it is not necessary, and probably not even a good idea, to study it exhaustively, at least not on a first reading.\ Part III is the real heart of the book. It consists of six chapters, one for each of the six sections of the Manifesto as defined in Part II. (Again, for the benefit of anyone who might have seen earlier drafts of the Manifesto, this part of the book consists essentially of a hugely expanded version of the informal commentary from those earlier drafts.) Each chapter discusses the relevant section of the Manifesto in considerable detail, with examples, and thereby explains the motivations and rationale behind the formal proposals of Part II (especially Chapter 3). Note, therefore, that the Manifesto itself serves as the organizing principle for this, the major part of the book.\ Finally, Part IV does for subtyping and inheritance what Parts I, II, and III do for the Manifesto proper. It consists of five chapters. Chapter 12 gives an overall introduction to the topic; Chapter 13 gives formal definitions; and Chapter 14 gives informal explanations and discussions of the ideas underlying those formal definitions. Chapter 15 then extends the ideas of Chapters 12-14 to address multiple inheritance, and Chapter 16 then extends those ideas further to take tuple and relation types into account as well.\ In addition to the foregoing, there are also several appendixes: one defining an alternative version of Tutorial D that is based on relational calculus instead of relational algebra, another discussing "subtables and supertables," another containing the text of an interview the present authors gave on the subject of the Manifesto in 1994, and so on. In particular, the SQL3 and ODMG comparisons can be found in this part of the book, as already mentioned. The final appendix, Appendix H, gives an annotated and consolidated list of references for the entire book.\ Note: While we are on the subject of references to publications, we should explain that throughout the book such references take the form of numbers in square brackets. For example, the reference "2" refers to the second item in the list of references in Appendix H: namely, a paper by Malcolm P. Atkinson and O. Peter Buneman entitled "Types and Persistence in Database Programming Languages," published in ACM Computing Surveys, Volume 19, No. 2, in June 1987.\ Finally, we should say a word about our use of terminology. It is our experience that many of the terms in widespread use in this field are subject to a variety of different interpretations, and that communication suffers badly as a result (examples seem superfluous—you can surely provide plenty of your own). While we have not deliberately used familiar terms in unfamiliar ways, therefore, we have nevertheless found it necessary to introduce our own terminology in certain places. We apologize if this fact causes you any unnecessary difficulties.\ Intended Readership\ Who should read this book? Well, in at least one sense the book is definitely not self-contained—it does assume you are professionally interested in database technology and are therefore reasonably familiar with classical database theory and practice. However, we have tried to define and explain, as carefully as we could, any concepts that might be thought novel; in fact, we have done the same for several concepts that really should not be novel at all but do not seem to be as widely understood as they ought to be ("candidate key" is a case in point). Thus, we have tried to make the book suitable for both reference and tutorial purposes, and we have indicated clearly those portions of the book that are more formal in style and are provided primarily for reference.\ Our intended audience is, therefore, just about anyone with a serious interest in database technology, including but not limited to the following:\ \ Database language designers and standardizers\ DBMS product implementers and other vendor personnel\ Data and database administrators\ Information modelers and database designers\ Database application designers and implementers\ Computer science professors specializing in database issues\ Database students, both graduate and undergraduate\ People responsible for DBMS product evaluation and acquisition\ People interested in any aspect of the "objects vs. relations" controversy, including especially anyone who might be involved in object/relational systems\ People interested in type theory and the topic of type inheritance\ For academic readers in particular (students as well as teachers), we should add that what we have tried to do is present the foundations of the database field in a way that is clear, precise, correct, and uncluttered by the baggage—not to mention mistakes—that usually (and regrettably) seem to accompany commercial implementations. Thus, we believe the book provides an opportunity to acquire a firm understanding of that crucial foundation material, without being distracted by irrelevancies. Perhaps we should say too that we believe there are several interesting (and, we hope, self-evident) research projects lurking just below the surface of certain portions of the material.\ \ Acknowledgments\ First of all, we are delighted to be able to acknowledge all of the numerous friends and colleagues who, over the past several years, have given encouragement, participated in discussions, and offered comments (both written and oral) on various drafts of The Third Manifesto or portions thereof: John Andrews, Tanj Bennett, Charley Bontempo, Declan Brady, Bob Brown, Rick Cattell, Linda DeMichiel, Vincent Dupuis, Bryon Ehlmann, Mark Evans, Ron Fagin, Oris Friesen, Ric Gagliardi, Ray Gates, Mikhail Gilula, Zaid Holmin, Michael Jackson, Achim Jung, John Kneiling, Adrian Larner, Bruce Lindsay, David Livingstone, Albert Maier, Carl Mattocks, Nelson Mattos, David McGoveran, Roland Merrick, Serge Miranda, Jim Panttaja, Mary Panttaja, Fabian Pascal, Ron Ross, Arthur Ryman, Alan Sexton, Mike Sykes, Stephen Todd, Rick van der Lans, Anton Versteeg, and Fred Wright (and we apologize if we have inadvertently omitted anyone from this list). We would also like to acknowledge the many conference and seminar attendees, far too numerous to mention individually, who have expressed support for the ideas contained herein.\ Second, we would like to thank our reviewers Charley Bontempo, Declan Brady, Rick Cattell, David Livingstone, and David McGoveran for their careful and constructive comments on the manuscript.\ Third, we are—of course!—deeply indebted to our wives, Lindsay Darwen and Lindy Date, for their unfailing support throughout this project and so many others over the years.\ Finally, we are, as always, grateful to our editor, Elydia Davis, and to the staff at Addison-Wesley for their assistance and their continually high standards of professionalism. It has been, as always, a pleasure to work with them.\ Hugh Darwen adds: My gratitude to my colleague and friend, Chris Date, goes without saying. However, I would like to comment on something, significant to us, that you possibly haven't noticed. It concerns the book's attribution. In our previous joint productions our names have been linked by the preposition with, intended to distinguish the primary author from the contributing assistant. This time around we have thought it more appropriate to use the conjunction and, of whose commutativity we Relationlanders are especially conscious! We came to this conclusion despite the fact that, as usual, Chris has done the lion's share of the actual writing. That the writing so faithfully and agreeably records our joint thinking (often painstakingly wrought out) is therefore a source of great pleasure to me, especially in those cases where I can still identify the thinking in question as having arisen from ideas first placed into discussion by myself.\ My own thinking has been molded, of course, with the aid of many valued mentors over the years, including Chris himself. Here I would like to single out just two other people for special mention: Adrian Larner for my relational thinking, and Nelson Mattos for my object-oriented thinking.\ Chris Date adds: If Hugh feels he has learned from me over the years, I can assure you (and him) that I have most certainly learned a great deal from him!—a state of affairs for which I will always be grateful. As for the matter of the book's attribution, it is of course true that The Third Manifesto is a joint effort, but Hugh should really take the credit for being the original and prime mover on this project: It was he who came up with the idea of the Manifesto in the first place, and it was he who wrote the very first draft, early in 1994. Though I should immediately add that our thinking on the matters with which the Manifesto deals goes back very much further than that; in some respects, in fact, I think we could claim that it goes all the way back to the beginning of our respective careers in the database field.\ \

PrefaceCh. 1Background and Overview3Ch. 2Objects and Relations15Ch. 3The Third Manifesto33Ch. 4A New Relational Algebra45Ch. 5Tutorial D61Ch. 6RM Prescriptions99Ch. 7RM Proscriptions171Ch. 8OO Prescriptions179Ch. 9OO Proscriptions187Ch. 10RM Very Strong Suggestions191Ch. 11OO Very Strong Suggestions219Ch. 12Preliminaries229Ch. 13Formal Specifications237Ch. 14Informal Discussions and Explanations243Ch. 15Multiple Inheritance299Ch. 16Tuple and Relation Types317App. AA Relational Calculus Version of Tutorial D343App. BThe Database Design Dilemma351App. CSpecialization by Constraint357App. DSubtables and Supertables373App. EA Comparison with SQL3379App. FA Comparison with ODMG409App. GThe Next 25 Years of the Relational Model?435App. HReferences and Bibliography455Index485

\ BooknewsA proposal for the future direction of data and database management systems. The focus is on providing a foundation for integrating relational and object technologies as a necessary evolutionary step. Includes explanations of many RM prescriptions and proscriptions, from scalar types and tuples to avoiding composite attributes and internal-level constructs. Also covers subtyping and inheritance. Appends a comparison with SQL3 and ODMG. Annotation c. by Book News, Inc., Portland, Or.\ \