Alert Classification and Prioritization

Our research in this area is intended to help organizations to better secure their code by using statistical methods to efficiently triage and prioritize static analysis alerts.

Those responsible for finding and fixing software flaws have a difficult task. Many techniques are used to find flaws. Automated static analysis tools examine the software without running it, and output warnings (i.e., alerts) about potential code flaws. Static analysis alerts and their associated code are manually examined by human analysts to make an audit determination about the alert (e.g., true or false positive). For most large codebases, there are too many alerts to address them all. If analysts and coders could prioritize alerts to know which to address first, their work would be more effective and efficient.

CERT alert classification and prioritization research aims to do just that. CERT researchers are continuing research in this area; goals for this research include improving classifier precision and accurately classifying more types of code flaws.

Our approach uses multiple static analysis tools on codebases. Studies (our own and others) have shown that single static analysis tools check for a subset of known code flaw types (e.g., CWEs or CERT coding rules). Using multiple static analysis tools means checking for more coding flaws, which compounds the problem of having a large number of alerts to address (both false and true positives).

We develop classifiers using alert audit archives, which contain data about alerts and their associated code, code flaws mapped to the alert (e.g., a CWE or CERT coding rule ID), and audit determinations. We unify outputs from multiple tools into one list and fuse alerts from different tools for the same code flaw at the same location. By developing and testing classifiers on archived data whose alerts have already been manually analyzed, we can test the precision of our techniques.

The goal with the classifiers is to precisely predict confidence in an alert being a true or false positive. The goal with prioritization is to optimally order alerts for manual inspection.

We are working with some new techniques that we can’t mention here yet, but we look forward to providing information about that new work soon (probably early summer 2017).