SCALe Conformance Testing

Meaning of "SCALe"

The name SCALe has been used historically to refer to a CERT code auditing process (provided as a part of a conformance testing service and sometimes used internally for research purposes), and refer to various versions of the code analysis alert auditing framework tool developed by CERT researchers. The SCALe auditing tool is used as part of the SCALe service. This webpage describes SCALe services.

SCALe Conformance Testing

Source Code Analysis Laboratory (SCALe) conformance testing uses commercial, open source, and experimental analysis tools to analyze software. It has been used to analyze software for the DoD, energy delivery systems, medical devices, and more. For each CERT secure coding standard, the source code for the software is certified as one of the following against each guideline in the standard.

  • Provably Nonconforming. The code is provably nonconforming if one or more violations of a rule are discovered for which no deviation has been allowed.
  • Conforming. The code is conforming if no violations of a rule can be identified.
  • Provably Conforming. The code is provably conforming if it has been verified to adhere to the rule in all possible cases.

Violations of a particular rule end when a provably nonconforming violation is discovered. Most SCALe analysis is performed by static analyzers. In general, determining conformance to coding rules is computationally undecidable. It may be impossible for any tool to determine statically whether a given rule is satisfied in specific circumstances.

The Recommended Resources on the right provide more detailed information about SCALe. However, be aware that the methods used for SCALe code conformance testing may have changed since these resources were published. Please contact the Secure Coding group if you have questions about the conformance testing options currently available or want to request a service.

The SCALe Conformance Process

A goal of conformance testing is to provide an incentive for organizations to invest in developing conforming systems by testing code against CERT secure coding standards, verifying that code conforms with CERT secure coding standards, using the CERT seal when marketing products, and maintaining a certificate registry of conforming systems.

When you request SCALe conformance testing, the following process is initiated:

  1. You request SCALe conformance testing and submit your source code for analysis.
  2. CERT staff analyzes the source code using various analyzers.
  3. CERT staff analyzes, validates, and summarizes the results.
  4. You receive a detailed report of findings to guide repairs to the source code.
  5. You address the identified violations and resubmit the repaired code.
  6. CERT staff reassesses the code to ensure that all violations were properly mitigated.
  7. Your certification for the product version is published in a registry of certified systems.

The CERT SCALe Seal

Developers of software that is determined by the CERT SCALe conformance testing to conform to a secure coding standard may use the CERT SCALe seal to describe their software on their website.

The seal must be specifically tied to the software passing conformance testing and not applied to untested products, the company, or the organization. Use of the CERT SCALe seal is contingent upon the organization entering into a service agreement with Carnegie Mellon University and upon the software being designated by the CERT Division as conforming.

Except for patches that meet the following criteria, any modification of software after it is designated as conforming voids the conformance designation. Until such software is retested and determined to be conforming, the new software cannot be associated with the CERT SCALe seal.

Patches that meet all three of the following criteria do not void the conformance designation:

  • The patch is necessary to fix a vulnerability in the code or is necessary for the maintenance of the software.
  • The patch does not introduce new features or functionality.
  • The patch does not introduce a violation of any of the rules in the secure coding standard to which the software has been determined to conform.

CERT SCALe Certificates

CERT SCALe certificates contain the name and version of the software system that passed the conformance test and the results of the test. This process is similar to that followed by The Open Group.

Initially, all assessments are performed by the CERT Division. In the future, third parties may be accredited to perform certifications.