Exploring Information Superiority: A Methodology for Measuring the Quality of Information and Its Impact on Shared Awareness

EXPLORING INFORMATION SUPERIORITY\ A Methodology for Measuring the Quality of Information and Its Impact on Shared Awareness \ \ By Walter L. Perry David Signori John E. Boon, Jr. \ Rand Corporation\ Copyright © 2004 RAND Corporation\ All right reserved.\ \ \ \ \ Chapter One\ INTRODUCTION \ It is the quality of our work that will please God and not the quantity.\ -Mahatma Gandhi\ The military is formulating new visions, strategies, and concepts that capitalize on emerging information-age technologies to provide its warfighters with significantly improved capabilities to meet the national security challenges of the 21st century. New, networked command, control, communications, computers, intelligence, surveillance, and reconnaissance ([C.sup.4]ISR) capabilities promise information superiority and decision dominance that will enhance the quality and speed of command and enable revolutionary warfighting concepts. Assessing the contribution of [C.sup.4]ISR toward achieving a network-centric warfare (NCW) capability is a major challenge for the Department of Defense (DoD) because of the multiplicity of interacting factors and the lack of understanding of the fundamentals associated with information-superiority concepts. DoD has embarked on a journey of exploration to discover how to create and leverage information superiority by characterizing the conditions under which it can be achieved and under which a competitive advantage can be gained. Much as in thedevelopment of a new branch of science, this requires defining concepts, metrics, hypotheses, and analytical methodologies that can be used to focus research efforts, identify and compare alternatives, and measure progress.\ In response to this need, DoD's Office of Force Transformation, in conjunction with the Office of the Assistant Secretary of Defense for Networks and Information Integration (ASD NII), has been evolving a conceptual NCW framework for assessment that includes measures, general forms for metrics, and relationships between the measures and metrics. The NCW Conceptual Framework is intended for exploratory analysis of potential [C.sup.4]ISR architectures and for guiding [C.sup.4]ISR experiments in terms of identifying the measures for which to collect experimental data.\ The technical, quantitative application of this framework requires developing mathematical models that reflect the performance of the enabling [C.sup.4]ISR processes, architectures, and capabilities, as well as developing a hypothesis identifying the key variables to measure, and their effects. Consequently, this report describes a methodology for developing and linking relevant mathematical models. It provides mathematical models corresponding to specific [C.sup.4]ISR architectures and illuminates the data needed to apply the NCW Conceptual Framework in a specific context.\ RESEARCH OBJECTIVES\ An important first step is to develop a better understanding of how improved [C.sup.4]ISR capabilities and related changes in command and control processes contribute to the achievement of such core NCW concepts as situational awareness, shared situational awareness, and synchronization. The next step is to assess how well the core concepts affect command and control operational concepts and, in turn, the success of military operations. Establishing a quantifiable link between improved [C.sup.4]ISR capabilities and combat outcomes has been extremely elusive and is therefore a major challenge.\ The primary objective of our work was to develop a clearly articulated mathematical framework that would explore how certain major factors would affect the hypotheses related to NCW. The scope of the research is restricted to the NCW Conceptual Framework's key measures related to information superiority, which Joint Publication 1-02 defines as "[t]hat degree of dominance in the information domain which permits the conduct of operations without effective opposition" (DoD, 2003, p. 255).\ In particular, we examined how the quality of information (individual and shared) a [C.sup.4]ISR network provides affects the quality of situational awareness (also individual and shared). In the NCW Conceptual Framework, the Quality of Information and Quality of Awareness are a vector of submeasures. The research focuses on a subset of these submeasures, notably completeness, correctness, and currency.\ The modeling approach was to study a chain of information quality and awareness quality metrics in three domains: physical, information, and cognitive. The models assumed that a specific type of [C.sup.4]ISR architecture is being employed, in which an array of sensors transmit data about the battlespace to a central fusion facility that distributes a resulting common relevant operating picture (CROP) to force members. The force members then mentally interpret the CROP they receive (creating awareness) and collaborate with other force members to improve everyone's awareness.\ This report emphasizes the development of the mathematical framework. Although we include several measures and mathematically defined metrics, they should not be considered the results of settled research. They are necessarily simple and illustrative rather than general and widely applicable.\ The quantitative methodology and illustrative mathematical representations here relate to force-on-force combat operations as opposed to a broader spectrum of operation that includes aid to civil authorities, humanitarian relief, and peacekeeping operations. However, aspects of the approach provide a basis for informed dialog that, in our view, will eventually lead to more-comprehensive and better-validated capabilities to quantitatively explore the influence of improved [C.sup.4]ISR systems and processes on operational outcome.\ ANALYTICAL FRAMEWORK\ In addition to the NCW Conceptual Framework, the research reported here builds on the work of the ASD NII Information Superiority Metrics Working Group (ISMWG). This body has developed working definitions, specific characteristics and attributes of key concepts, and the relationships among them that are needed to measure the degree to which information superiority concepts are realized and their impact on the conduct and effectiveness of military operations. Such an endeavor requires a common language and a set of integrated hypotheses, as well as the metrics, instruments, and tools to collect and analyze data, such as those suggested here.\ We begin by defining a reference model for discussing such issues in terms of three domains: that of ground truth (the physical domain); that of sensed information (the information domain); and that in which individual situational awareness, shared situational awareness, collaboration, and decisionmaking occur (the cognitive domain). The [C.sup.4]ISR process is seen as extracting data from ground truth and processing the data in the information domain to produce a CROP. The quality of the CROP combines with the quality of team collaboration to heighten (or degrade) shared situational awareness in the cognitive domain. The ground truth data, obtained from the collection process in the information domain, is transformed into a CROP that contributes to situational awareness in the cognitive domain. The transformations are processes that include data collection and processing, data fusion, and information dissemination. These processes are not discussed in this report. We assume that they are performed and focus instead on the quality of the information and products they generate.\ LIMITATIONS\ This report focuses on the collection of data; the processing of collected data to produce the CROP; the dissemination of the CROP from the various fusion facilities to the ultimate users; the experience of the decisionmaking team; the quality of its collaboration; and, finally, how all this affects shared situational awareness. In developing mathematical representations of information quality and the effects of collaboration, we focused exclusively on battlefield entities -individual units and weapon systems. The possible inferences to be drawn from the patterns these entities exhibit, although very important, are beyond the scope of this initial work.\ We also do not treat several important aspects of the decisionmaking cycle, such as decisionmaking and synchronization. The relationship between awareness and understanding and how understanding affects decision and action are also not addressed. Nevertheless, this document presents a new and important methodology for assessing the quality of information and intelligence, surveillance, and reconnaissance processes in general, as well as some of the more psychological aspects of the decisionmaking process. It is based on sound mathematical concepts and hypotheses from the literature and therefore provides a foundation for further inquiry. The illustrative mathematical relationships, however, will require verification and refinement through experimental and operational data.\ Finally, with respect to the NCW Conceptual Framework, this report is restricted to the information and awareness measures. The report does not attempt to show models that traverse all the measures in the framework.\ ORGANIZATION OF THIS REPORT\ Chapter Two outlines the analytic framework we adopted to assess the effects of information quality and team collaboration on shared situational awareness and, eventually, decision and execution. We begin by describing the [C.sup.4]ISR Information Superiority Reference Model as the underlying construct that describes the [C.sup.4]ISR process. Chapter Three focuses on a mathematical framework for assessing the contributions of both the physical and information domains to information quality. Chapter Four turns to the more-psychological aspects of decisionmaking and assesses the contributions of the cognitive domain to shared situational awareness. Finally, Chapter Five addresses future work. Three appendices are included: Appendix A lists the definitions of important terms; Appendix B records a few example mathematical representations for the metrics developed in the main text; and Appendix C describes a spreadsheet model used to illustrate the framework using the example metrics listed in Appendix B.\ Chapter Two\ THE ANALYTIC FRAMEWORK \ A decision is an action an executive must take when he has information so incomplete that the answer does not suggest itself.\ -Arthur William Radford\ This chapter outlines the analytic framework we adopted to assess the effects of information quality and team collaboration on shared situational awareness and, eventually, decision and execution. We begin by describing the underlying construct that describes the [C.sup.4]ISR process we model here: the [C.sup.4]ISR Information Superiority Reference Model. The quality of the information flowing through what we call "the information value chain" is transformed to produce an overall assessment of the quality of the CROP generated. This, in turn, affects individual situational awareness and, subsequently, shared situational awareness through the process of team collaboration.\ THE [C.sup.4]ISR INFORMATION SUPERIORITY REFERENCE MODEL\ The [C.sup.4]ISR Information Superiority Reference Model (Figure 2.1) is a representation and extension of the portion of the NCW Conceptual Framework dealing with information superiority issues (in particular, the information and awareness measures). It represents the activities associated with collecting data; processing information; discerning enemy intent, plans, and physical activities (or lack thereof); selecting a course of action; and monitoring its execution. The model consists of three "domains" that extend from the battlefield environment to cognitive awareness of the battlefield situation and decision. The measures and metrics proposed here, although not all-inclusive, are presented within this context.\ Both sides in a conflict generally have different perceptions of a single reality, referred to as the situation. Figure 2.1 depicts how the three domains contribute to this perception. The major activities performed in each of the domains are listed in its box. The physical domain is where reality or ground truth resides.\ In addition to physical objects (entities)-weapon systems, terrain features, sensors, etc.-the physical domain also contains intangibles, such as enemy intent, plans, and current and projected activities. A complete assessment of the situation will contain estimates about each. As mentioned earlier, however, we will focus on entities only, thus reserving the others for future research.\ In the information domain, data are extracted from the physical domain and processed to form structured information in the form of a CROP. Three primary functions are performed in the information domain: collecting data through the use of sensors and sources, including tasking sensors to close gaps in the data; processing the data through the fusion process to produce the CROP; and disseminating relevant parts of the CROP to friendly units. The last step contributes to the collaboration process in the cognitive domain, in which the shared CROP is transformed into a shared awareness of the current and future situations that can be used to gain understanding of threats and opportunities, as well as the subsequent decisionmaking regarding an appropriate course of action.\ Finally, the human activities associated with using the information available to form an estimate of the situation take place in the cognitive domain. To the extent that decisionmaking teams exist, they collaborate to form a level of situational awareness. In addition to the CROP produced in the information domain, individual team members and the decisionmaker may have prior information from such processes as the information preparation of the battlefield (IPB) available to support their deliberations. Finally, the decisionmaker is likely to have concerns and expectations about the performance of his own forces, as well as those of the enemy, that would color his assessment of the situation and therefore his decision. These are depicted as emanating directly from the physical domain.\ THE [C.sup.4]ISR ARCHITECTURE\ Employing the NCW Conceptual Framework and the Information Superiority Reference Model requires a [C.sup.4]ISR architecture. The architecture used in this report may be thought of as a linear process with six steps:\ 0. Ground Truth. The architecture begins with acceptance of the existence of physical ground truth. As the previous chapter noted, we restricted our consideration of ground truth to battlefield entities, such as individual units and weapon systems, and their physical attributes.\ 1. Sensing. The architecture first uses a set of sensors to detect the battlespace entities and their sensors. 10 Exploring Information Superiority\ 2. Fusion. The sensors then transmit their data to fusion facilities. The fusion structure used in this report is centralized and has two stages. In the first stage, incoming sensor data are sent to one of k fusion facilities, each corresponding to a different intelligence discipline. In the second stage, the partially fused data from each discipline are forwarded to a central processing facility, which generates a single CROP. For the sake of simplicity, we assumed that the network connections transmitting data from the sensors to the fusion facilities are flawless.\ 3. Distribution. The central processing facility then transmits various versions of the CROP to the network's users. At this point, we allowed the modeling of loss of service, errors, and delays in the network.\ 4. Individual Assessment. Each user then attempts to interpret the CROP he has received to achieve some level of realization of the battlespace.\ 5. Group Assessment. The users then collaborate with each other in an attempt to improve their realization of ground truth in the battlespace. This report models the effectiveness of collaboration as a function of the skills of the users and the collaborative group as a whole but does not examine the affects of the network's communications tools on collaboration.\ (Continues...)\ \ \ \ \ Excerpted from EXPLORING INFORMATION SUPERIORITY by Walter L. Perry David Signori John E. Boon, Jr. Copyright © 2004 by RAND Corporation. Excerpted by permission.\ All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.\ Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site. \ \