Disaster Recovery Planning: Preparing for the Unthinkable

Author's Preface to the Third Edition\ On reading the forewords to this edition, which consist of first hand accounts of the events that occurred during and after the September 11, 2001 terrorist attacks on the World Trade Center, one cannot help but be moved—and somewhat awestruck.\ In addition to the sadness and tragic loss associated with 9/11, we are also magnetized by the remarkable tenacity and heroism manifested by those who worked to save lives and to recover organizations in the hours, days, and weeks following the disaster. This edition is dedicated to spirit of the survivors and to the memory of the victims.\ When my editor approached me to write a third edition of this book in the aftermath of the September incidents, my first reaction was one of hesitance. Was a new edition really justified, or was it simply a marketing ploy designed to capitalize on the current surge of interest in disaster recovery and risk management in the wake of 9/11? To keep things honest, I performed a quick inventory:\ Question: Had September 11 changed what we know about disaster recovery?\ Answer: Of course not. The methodologies and procedures and best practices that comprise disaster recovery planning did not change in the wake of this latest disaster any more than they had changed following Hurricane Andrew or the Kobe earthquake. If anything, 9/11 had confirmed the efficacy of planning and, once again, demonstrated the difficulties of undertaking recovery successfully in the absence of proactive DR planning.\ Question: Is terrorism a new threat to organizations that requires new measures of preparedness?\ Answer: Again, no. The terrorist threatpotential has been an unfortunate fixture in the disaster scenarios of private and public institutions for many years—both in the United States and abroad. Moreover, disaster recovery planning is less concerned about the root cause of disasters (unless such knowledge can help disasters to be averted in the first place), than in coping with their consequences. From a disaster recovery standpoint, it mattered very little whether 9/11 was caused by Al Qaeda operatives, bin Laden fanatics, or insane hijackers of any political or religious stripe. The disaster might just as well have been the result of an electrical spark or other igniter.\ What was important to disaster recovery planners was how the most critical assets of the organizations located in the WTC and the Pentagon—specifically, the trained personnel and the data—would be protected and returned to work quickly and efficiently in the wake of the disaster. Simply put, whatever the toll of the disaster, failing to respond to the events in a rational way would have made the costs much greater.\ Question: Did the events of 9/11 change the milieu in which disaster recovery plans must execute?\ Answer: Maybe. This event was not a manifestation of natural forces, nor a consequence of random circumstances. It was a deliberate act by hostile force that has touched off a response increasingly characterized as a war. As a consequence, it has placed certain societal and governmental institutions on a war footing. That does create a meaningful change in the milieu in which disaster recovery plans will need to be executed.\ Without a doubt, new legal mandates will follow the events of 9/11. Some involve the "hardening" and protection of public infrastructure. Security has already been stepped up within the energy and transportation sectors. Information and communications are also coming under greater scrutiny as investigations turn to discovering how the terrorists obtained detailed intelligence on structural vulnerabilities and how they were able to establish false identities with such apparent ease.\ When milieu changes occur, disaster recovery plans need to be re-tested. So too with books on disaster recovery: assumptions need to be re-visited and reconfirmed.\ Question: Aside from 9/11, have any other technical changes occurred that merit revision of the content of this book?\ Answer: Yes. Ten years elapsed between the first edition (1989) and the second edition (1999) of Disaster Recovery Planning, during which information technology moved from the data center and into the distributed environment of departments and workgroups within many organizations. The second edition endeavored to update the content of the original to reflect the new challenges and opportunities created by this shift.\ In the comparatively brief period of time between 1999 and 2002, one could argue that an equally important technology shift has occurred. Several indicators testify to the truth of this assertion.\ Data Growth: According to a study conducted by the University of California at Berkeley, the amount of data amassed by organizations and stored electronically in all of human history through 1999 totaled twelve exabytes (12,000,000,000,000,000,000 bytes). Researchers observed, however, that this number would double by mid-2002—a function of greater amount of information generated by a greater number of individuals.\ The bulk of burgeoning data is being created by individuals and the lion's share, some 55 percent, is being stored on personal computers that are often not included in disaster recovery planning efforts. This is potentially an Achille's heel of disaster recovery—a point underscored by many 9/11 recovery efforts.\ New Storage Technologies: Managing the data explosion referenced above—providing for its secure yet accessible storage—has become the central challenge confronting IT in the 21st century. Today, we are seeing the advent of nascent, "networked," storage technologies that portend to sever the connection between storage arrays and servers. Such technologies are expected to improve data accessibility and to provide nondisruptive scalability. Vendors claim that their wares also have potential for improving the resiliency of data storage and for reducing the timeframe for critical business process restoration, which is increasingly described by the metric "time-to-data." The truth of these assertions has yet to be demonstrated and interoperability problems continue to plague solutions, causing many more disasters than they prevent.\ New Application Paradigms to Support Business Process Deconstruction: Currently, the industry is seeing the rise of new technologies such as\ Such technologies are required to support the larger trend toward "business process deconstruction"—a means by which companies improve the efficiency of business processes by outsourcing more logistical responsibility to supply chain and value chain partners. The question is whether the still-evolving technologies will cause more disasters than they avoid. Will the use of ASPs increase the vulnerability of business processes to disaster or reduce it? Will application Web-enablement deliver less-costly and more secure B2B operations or simply succeed in making multi-tier client/server platforms even less stable than ever before?\ The above list could go on, but by the conclusion of this brief inventory, I determined that a new edition of this book was indeed merited. A new edition was mandated not by the events of 9/11 directly, but by changes in the information technology infrastructure that supports key business processes and by changes in the organizational and external milieu in which disaster recovery plans must execute. It can be argued (and will be) that these changes require that the traditional approach to disaster recovery planning must also change.\ Rather than being reactive, "playing the hand of cards they are dealt," DR planners will need to become more proactive in their efforts. They will need to begin to interact with application architects on an ongoing basis and to begin addressing the recoverability of applications, storage, and IT infrastructure while they are still in their initial design phases. DR planning can no longer be conceived as a "bolt on" or an afterthought: it must become an integral part of systems development.\ Of course, to move into this role, planners themselves will need to become more technically astute. The days of "secretary friendly" plans are increasingly behind us. To interface with technical personnel—IT designers and architects, in particular—planners will need to become more proficient in the concepts and terminology of advanced technology domains. The modern planner will need to be conversant in object-oriented programming, middleware, extensible markup language, storage area networking, and many other rarified facets of information technology. To walk the walk of the IT architect, planners will need to talk the talk.\ Planners will also need to become more business savvy. While it is a given that 9/11 has increased business awareness of the need for DR, history teaches us that business interest in DR has a tendency to wane over time. It is only natural: just as current events eventually become footnotes in history books, the more time that elapses following a disaster event, the less urgency business managers tend to place on disaster preparedness. Practicality dictates that companies spend money on projects that advance the goal of profitability.\ Thus, business savvy planners will contextualize their efforts using a more robust business value proposition than mere "risk reduction." Dual use—the philosophy of delivering DR solutions that both reduce risk and enable business in normal day-to-day operations—must be paramount in DR strategy development going forward.\ Ultimately, disaster recovery planning will need to become an integral part of the business and IT decision-making process. The development of a new business process and selection of every element and component of the technology infrastructure to support that business process should have recoverability as a key criterion.\ When this happens, the discussion of disaster recovery planning as a separate discipline will be oxymoronical. Disaster Recovery Planning will no longer be a role for a select individual or dedicated cadre of planners. It will be the job of everyone in the organization—from the most senior manager to the most junior staff member, from the business professional to the IT professional.\ And, of course, the need for new editions of this book will fade away. (I, for one, would welcome this outcome.)\ But, until such a day arrives, new editions of Disaster Recovery Planning will be forthcoming to address the challenges of change in business, technology, and the backdrop of institutions and cultures in which we must all operate.

ForewordForewordAuthor's Preface to the Third EditionPreface to the Second EditionAcknowledgmentsCh. 1Introduction1Ch. 2Analyzing the Risk35Ch. 3Facility Protection63Ch. 4Data Recovery Planning120Ch. 5Strategies for Centralized System Recovery220Ch. 6Strategies for Decentralized System Recovery255Ch. 7Strategies for End User Recovery279Ch. 8Strategies for Networking Backup299Ch. 9Emergency Decision Making352Ch. 10The Recovery Management Improvement397Ch. 11Plan Maintenance and Testing424Ch. 12Conclusion439Glossary455Index