Evaluating Incident Management Capabilities
As CSIRTs and other incident management capabilities mature, it is beneficial for organizations to evaluate how well they are meeting their missions and conducting their operations in an effective and efficient manner. Any evaluation criteria or mechanism should be done with management approval and collaboration.
Evaluations can be performed for a variety of reasons to meet different criteria including
- incident handling satisfaction
- incident response timeliness
- damage from an incident
- process workflow
- general mission success
Organizations can choose different measures including
- benchmarking against other organizations or established standards
- interviews and discussions with constituency representatives
- evaluation surveys
- audits or third-party evaluations based on predefined quality parameters
The CERT CSIRT Development team has the following two methods that organizations can use to evaluate and improve their capability for managing computer security incidents.
Incident Management Capability Metrics (IMCM)
The IMCM provides organizations with a baseline against which they can benchmark their current incident management processes or services. More details about IMCM are available in the SEI technical report, Incident Management Capability Metrics.
The goal of this incident management capability evaluation is to help organizations assemble the right set of people, processes, and technology that enables them to protect and sustain their critical data, assets, and systems, and to conduct appropriate response and coordination actions for handling events and incidents when they occur. These metrics can be used to
- evaluate an existing capability
- identify areas for process improvement in an existing capability
- help determine the services and functions needed to create an incident management capability
The results obtained from the IMCM help an organization determine the maturity of its incident management capability regardless of organization or sector type (commercial, academic, government, etc.).
Mission Risk Diagnostic for Incident Management Capabilities (MRD-IMC)
The Mission Risk Diagnostic for Incident Management Capabilities (MRD-IMC) is a risk-based approach for assessing the extent to which an incident management function is in position to achieve its mission and objectives. The SEI technical note An Introduction to the Mission Risk Diagnostic for Incident Management Capabilities (MRD-IMC) provides more details on this approach. Analysts applying the MRD-IMC evaluate a set of systemic risk factors (called drivers) to aggregate decision-making data and provide decision makers with a benchmark of an incident management function’s current state. The resulting gap between the current and desired states points to specific areas where additional investment is warranted. The MRD-IMC can be viewed as a first-pass screening (i.e., a “health check”) or high-level diagnosis of conditions that enable or impede the successful completion of the incident management function’s mission and objectives.
Other research in the area of evaluating incident management capabilities is being done by the CSIRT Metrics Special Interest Group (SIG) at FIRST. More information on that work can be found on the FIRST website.