Key Message: Implementing secure coding standards to reduce the number of vulnerabilities that can escape into operational systems is a sound business decision.
Executive SummaryAny software defect, whether security related or not, that is allowed to escape into a production system can cause a disruption of service affecting thousands to millions of users. Costs to both providers and users can be staggering – up to millions of dollars. About 3 years ago, Cisco decided to develop and implement a Secure Development Lifecycle Methodology (CSDL). The CSDL includes secure coding guidelines based on the CERT Secure Coding Standards. While local solutions had been in use for some time, a key goal of this effort has been to ensure a coding standard that applies uniformly across the entire organization to achieve consistency in Cisco’s software quality and security. The standard is enforced and has automated support through the use of, for example, compilers and static analysis tools.
In this podcast, Martin Sebor, a technical leader at Cisco Systems, discusses Cisco’s adoption of the CERT Secure Coding Standards and why the use of such standards is an essential part of Cisco’s software development process.
Martin is a member of Cisco’s Compiler Tool Chain team, responsible for developing:
Martin interfaces with the Advanced Security Initiatives Group (ASIG) on security-related projects. The ASIG supports Cisco’s Secure Development Lifecycle Methodology (CSDL), which was put in place about 3 years ago.
Until recently, the CSDL was missing a set of consistent security coding practices across the entire enterprise. Up to this point, there were localized solutions, the majority of which relied on informal enforcement via the code review process, with little automated support.
Cisco decided on a phased approach to integrate security into their development process. Martin became involved when the CSDL methodology and development tools – compilers and static analyzers – were ready for the formal introduction of a secure coding standard.
Martin had been involved with the CERT Secure Coding Standards project and participated in the C and C++ international standardization committees, so was appointed to lead Cisco’s efforts in this area.
A key goal was to ensure that the Cisco standard would apply uniformly across the entire organization to achieve consistency in their software quality and security.
Cost vs. BenefitAny defect, whether security related or not, that is allowed to escape into a production system can cause a disruption of service affecting thousands to millions of users. Such a defect may also violate service level agreements. So costs to both providers and users can be staggering – up to millions of dollars.
Finding Defects During System TestingFinding a defect during system testing certainly helps. However, vigorous testing is an involved and time-consuming process, taking weeks and possibly months for a complex system.
When a vulnerability-inducing defect is discovered during system testing, the software development team has to fix the defect. And then the software must be re-tested. This is costly to both the development and test teams.
It is not uncommon to incur costs of hundreds of thousands of dollars as well as incur costs resulting from schedule delays.
Finding Defects During Unit TestingFinding a defect during unit testing is somewhat better. But such defects may cause cascading failures across interfacing components. Such failures can render component test results useless. It is often challenging to find the root cause of the underlying defect.
Finding Defects During Coding: Automation Is EssentialThe best time to uncover a coding defect is when it is injected into the code during development. Visual inspections such as code reviews is one effective mechanism. However, a more reliable approach is the use of automated tools such as compilers and static analyzers.
Ideally the defect is discovered before the code changes in the shared code base so it doesn’t affect other developers.
When this happens, the code author simply corrects the code and reruns the tool that revealed the defect to confirm it has been corrected. This is much more cost-effective.
Cisco considered a number of internal coding standards as well as external standards developed by other organizations.
They chose the CERT Secure Coding Standards for the following reasons:
This is the approach that was used at Cisco. It may vary slightly, or substantially, depending on the organization, its size, and its culture:
Software engineers tend to think first about functionality and performance. They introduce assumptions into the code base based on the "happy path” – the expected path through their programs.
It takes a change in mindset to appreciate that the happy path is likely not the path that will be taken by an attacker. This change can be accomplished by demonstrating examples of vulnerabilities and the attacks that exploit them, during tech talks or brown bag lunch meetings.
Link Exploited Vulnerabilities to Ineffective Secure Development PracticesA promising direction for future work is to examine the extent to which exploited vulnerabilities result from absent or ineffective secure coding practices, as well as violations of in-place coding standards as a root cause contributor.
ResourcesCERT’s Secure Coding Standards website and wikis
Cisco’s Secure Development Lifecycle Methodology (CSDL)
CERT podcast: Mainstreaming Secure Coding Practices, March 2009.