An attribute such as survivability does not exist in isolation. A system has multiple quality attributes such as performance, availability, and modifiability. Attributes and their analyses interact. Performance affects modifiability. Availability affects safety. Security affects performance. Everything affects cost. While experienced system designers know that these tradeoffs exist, no codified method exists for characterizing quality attributes and, in particular, characterizing their interactions.

Often, system designers neglect to consider survivability and security in their designs. Typically, survivability is considered in isolation from other attributes of software-engineering quality, such as performance, dependability, modifiability, and ease of use. This approach is not surprising, since in traditional software engineering the other quality attributes dominate the design process whereas, survivability and security (if considered at all) are usually an afterthought.

Software designers, their managers, and their customers must be able to specify the tradeoffs between enhanced survivability and other attributes of software quality (including affordability). These groups also must be able to evaluate how well competing designs (and implementations) achieve overall system specifications, including survivability specifications.

Survivability and security typically are relegated to a series of add-on patches that are put in place after problems are discovered and reported either by the customer or by an incident response team. Because these patches are typically quick reactions to an emergency situation, rather than the result of principled systems engineering design, they do not solve broad classes of problems. These patches often solve only a small number of problems and leave many others unsolved. In fact, they occasionally introduce new security vulnerabilities (or bring old ones back to life).

When a patch is designed, the development focus is often on the speed and ease of the solution, and on maintaining performance and functionality (e.g., don't degrade existing abilities) rather than on designing the best solution from a survivability or security standpoint. Conversely, those designing a patch sometimes decide to maximize security and survivability at any cost. Using either approach, leads to design efforts in which tradeoffs are not appropriately considered.

Changes to software systems are rarely made at the architectural level. For example, the patches for security vulnerabilities are not usually reflected in the architectural description of the system. This disconnect between the architecture and the implementation of a system frequently has an unexpected and undesirable impact on security and other attributes of survivability. Daily, the CERT Coordination Center sees the real-world damage (system intrusions) caused by the lack of a theoretical foundation upon which to build sound software engineering practices for the design, implementation, and maintenance of survivable systems.

The characteristics and dimensions of the design tradeoffs that arise once survivability is made an inherent part of the software-engineering process is a fertile area for exploratory and applied research. This area has the potential for a high payoff in software engineering process improvement.

Survivability issues affect system requirements in several areas. In Section 2, we identified four requirements categories of system and survivability functions:

• usage, including intrusion use
• development methodology
• system operation
• system evolution

In this section, we concentrate on an approach that addresses the first two requirement categories, specifically, how to evaluate the capability of a system to deliver essential functions in an environment that includes intrusion scenarios. Our general approach to survivability and security is consistent with the SEI-developed Architecture Tradeoff Analysis (ATA) [Kazman 97].

The ATA approach is based on

• a set of system attributes
• analytic measures of the system that are based upon formal models (e.g., performance and availability)
• qualitative measures of the system that are based upon formal inspections (e.g., modifiability, safety, and security)

Each of these measures evaluates the software architecture along a distinct dimension. Taken together, these dimensions are of interest to the system’s stakeholders. Figure 5 illustrates the ATA process. The scenario-based Software Architecture Analysis Method (SAAM) applied to modifiability and extensibility is described in [Kazman 96].

Scenarios are a way to evaluate a proposed architecture before the system is built. For intrusion requirements, this evaluation involves generating possible attack scenarios and evaluating the capability of the system to resist, recognize, and recover from such attacks, and to adapt and evolve so as to limit the effectiveness of future attacks.

A scenario is not necessarily an explicit script for breaking into a system. Scenarios can focus on the impact of having a critical system component, such as user authentication, compromised and on how to recover once that situation occurs. Generating such scenarios requires looking at the system from the intruder’s perspective. Intruder’s often have limited resources and their strategies often depend on

• exploiting known weak spots in technology
• identifying system dependencies and weak links
• monitoring network communication between components to capture information, monitor activity, and disrupt communications

Scenarios should identify system hot spots that provide opportunities for successful intrusions. This hot-spot method is a practical approach ideally suited for contingency planning to combat an intelligent adversary. Contingency planning includes analyzing survivability for as many intruder scenarios as practical. Scenarios also document those aspects of a bounded environment that the system is designed to confront.

The scenarios should account for intrusion objective, impact, strategies, and properties.

The intent of intrusion can include denial of service, access to confidential data, or compromise of existing data.

The impact of intrusion on a system can be direct or indirect. Direct impact can affect performance or availability. Indirect impact can result in loss of business and customer trust. Data collected on intrusion impact is part of the cost-benefit analysis used in the tradeoff analysis of system properties.

An intrusion strategy is a technique that helps intruders achieve one or more intrusion objectives. The set of intrusion strategies is unlimited. The combination of the strategies that an intruder chooses reflects the specific objectives and knowledge of that intruder. Intrusion strategies include established techniques and new approaches.

Intrusion properties are quantitative properties that systems exhibit when they are successfully penetrated and compromised. Each property can be defined by the observable system effects it produces. The set of properties is unlimited. The combination of properties that an intruder chooses for given set of intrusion requirements reflects the specific objectives of that intruder. Some properties, such as root privileges, may be achieved by exploiting system vulnerabilities, or by other means, such as compromising a system administrator.

The ATA process evaluates an architecture using a collection of system properties. The tradeoff analysis regarding security and survivability consists of several tasks:

• Generate intrusion scenarios.
• Establish priorities and costs associated with the effects of the scenarios.
• Identify architecture hot-spots based on effects of the intrusion scenarios.
• Generate and evaluate security and survivability strategies for each proposed architecture regarding the hot spots.
• Identify the requirements for system-level services such as user authentication and authorization that may be necessary for security or survivability.

Some hot spots will be associated with known vulnerabilities of the proposed technologies. A strategy to address this type of situation may be an architecture that supports an alternate implementation of a service (e.g., Ethernet and Token Ring for a local area network) or permits the rapid upgrade of a service, such as a Web server, to fix a new vulnerability.

Hot spots may be associated with the general communications topology of the system. For example, a central database or directory service may maintain user profile and authentication information. There may be specific strategies to protect that information and restrict system access if there are communications problems.

Hot spots may also be associated with application logic and the protocols used to exchange information among distributed components. As we discussed in Section 3, there is limited global knowledge that is available about this subject. Analysis should concentrate on pair-wise exchanges of data. Components that maintain extensive state information may complicate recovery.

Figure 6 summarizes an architectural evaluation process for survivability based on the principles of architecture evaluation through system-use scenarios. This evaluation process regards intruders as another class of user, on par with the legitimate users of a system. This process is an extension and specialization of the ATA process described in Figure 5.

The evaluation process permits analysis of the interaction between survivable architectures and intrusion environments through three major activities: survivable architecture definition, intrusion environment definition, and survivable architecture tradeoff analysis.

Survivability functions can be defined and embedded within candidate architectures based on survivability requirements and strategies. These architectures also include system functions that are required by users.

Intrusion capabilities can be defined and embedded within system-use models based on intrusion requirements and strategies. These system-use models also include the uses of system functions that are required by users.

The performance of candidate architectures can be analyzed using scenarios that are generated by the system-use models. This analysis results in feedback to requirements and possible modifications to architecture and system-use definitions for subsequent analysis.

We plan to evaluate and report on the effectiveness of this evaluation process by applying it in several pilot studies.