Safety and security hazards have many similarities. Both deal with threats or risks. Safety primarily deals with threats to life or property while security has traditionally dealt with threats to privacy or national security. With the current interest in information warfare and security threats to the critical infrastructure, this difference is fading. Both often involve negative requirements or constraints that may conflict with some important system goals. Both involve protection against losses, although the types of losses involved may be different. Both involve global system properties that are difficult to deal with outside of a system context. Both involve requirements that are considered of supreme importance (in relation to other requirements) in deciding whether the system can and should be used-that is, particularly high levels of assurance may be needed, and testing alone is insufficient to establish those levels.

     They also differ in one important way. Security focuses on malicious actions, whereas safety is primarily concerned with well-intended actions.

     Despite this difference, we believe that the application of software safety techniques to system security and vulnerability has promise. In particular, two techniques we have developed for safety seem to be directly applicable to survivability: Intent Specifications and Software Deviation Analysis.

Intent Specifications
     The first technique that we believe could be usefully applied to survivability is a new way of structuring specifications called Intent Specifications. Instead of the usual hierarchical abstraction based on what and how, Intent Specifications abstract on intent or why. Because each level is mapped to the appropriate parts of the intent levels above and below it, traceability of design rationale and design decisions is provided from system-level requirements and constraints down to code (or physical form if the function is implemented in hardware) and vice versa.

     Intent specifications integrate formal and informal aspects of system and software development. The structure is designed to facilitate the tracing of system level requirements and constraints into the design and assist in the assurance of various system properties-such as safety, security, and survivability-in the initial design as well as reduce the costs of implementing changes and reanalysis when the system is changed, as it inevitably will be. Leveson has applied these ideas to specifying control systems with shared human and computer control and to assuring safety in these systems. Leveson and Reese then demonstrated the approach's feasibility by creating a complete (800 page) system specification (from high-level system goals down through code) for TCAS II, an aircraft collision avoidance system.

Software Deviation Analysis
     The second approach we believe could be applicable is a technique called Software Deviation Analysis (SDA) [1, 2]. SDA is a new safety analysis technique to help identify weaknesses in how software handles an imperfect environment. The technique propagates deviations in software inputs to output deviations. A qualitative analysis is used to improve the search efficiency.

     SDA [3] is based on the underlying systems theory that accidents are caused by deviations in system parameters. A deviation is the difference between the actual value of a system variable and the expected (or "correct") value. Examples of deviations are "too high" and "too low." SDA analyzes the effect of deviations in software inputs on software output.

     The basic inputs to the SDA algorithm are a formal software requirements specification, a list of safety-critical software outputs, and some initial assumptions about the values of the software inputs, including at least one deviation in these inputs.

     The output of the SDA algorithm is a list of scenarios. A scenario is a set of deviations in the software inputs plus constraints on the execution states of the software that are sufficient to lead to an output deviation in a significant variable.

     Figure 1 shows an overview of the SDA process. The analyst provides a formal software requirements specification, which the procedure automatically converts into a more basic representation, called a causality diagram. The causality diagram is an internal data structure that encodes causal information between system variables, based on the specification and the semantics of the language in which it is written. The simplicity of causality diagrams makes the search algorithm more straightforward and easier to adapt to a new specification or programming language.

Figure 1: Overview of SDA Process

     Each node in the specification is linked to its corresponding node in the causality diagram, so that the results of the analysis can be translated from the causality diagram back into the language of the specification, avoiding the need for the analyst to comprehend or even see the causality diagram.

     The procedure uses deviation formulas, which define how deviations are related. This information is incorporated directly into the causality diagram to create an augmented causality diagram.

     SDA uses qualitative mathematics on the augmented causality diagram to evaluate deviations. Qualitative mathematics partitions infinite domains into small sets of intervals and provides mathematical operations on these intervals. The use of fixed intervals simplifies the analysis compared to iterations over the entire state space. It also lends itself naturally to the qualitative nature of deviations, such as "slightly too high."

     The augmented causality diagram, input deviations, and a list of safety-critical variables is passed to the search algorithm, which constructs a tree of states. The state formed by the input deviations is the root of the search tree. Leaves are either dead-end searches (in which a state does not contain any deviations) or states containing safety-critical deviations. The output of the SDA procedure is a list of the paths from the root state to all leaves with safety-critical output deviations.

     In summary, SDA may be thought of as a type of symbolic execution: The specification is used to propagate symbols representing classes of values (as defined by the calculus of deviations). It may also be thought of as a limited theorem prover that provides derivations of the conditions that can lead from an initial state to a hazardous state.

     We have applied SDA to several realistic systems. Although more extensive experimentation needs to be performed, the procedure appears to provide useful information for requirements specification and review.

     To conclude, we believe there is great potential in Intent Specifications, Software Deviation Analysis, and other safety analysis techniques in the field of critical-systems survivability. Our experience with safety critical systems and formal modeling of industrial size systems would be a benefit to the program at IWS'98.

[1]

J.D. Reese and N.G. Leveson. Software Deviation Analysis: A "Safeware" Technique. AIChe 31^st Annual Loss Prevention Symposium, Houston, TX March 1997.

[2]

J.D. Reese and N.G. Leveson. Software Deviation Analysis. International Conference on Software Engineering, Boston, May 1997.

[3]

J.D. Reese. Software Deviation Analysis. Ph.D. thesis, University of California, Irvine, 1996.

Back to the Table of Contents
   [26]