SEI CERT C++ Coding Standard

Introduction - Conformance Testing
• False negatives: Failure to report a real flaw in the code is usually regarded as the most
serious analysis error, as it may leave the user with a false sense of security. Most tools err on
the side of caution and consequently generate false positives. However, in some cases, it may
be deemed better to report some high-risk flaws and miss others than to overwhelm the user
with false positives.
To the greatest extent feasible, an analyzer should minimize both false positives and false
negatives with respect to enforceable guidelines.
Compilers and source code analysis tools are trusted processes, meaning that a degree of reliance
is placed on the output of the tools. Accordingly, developers must ensure that this trust is not
misplaced. Ideally, trust should be achieved by the tool supplier running appropriate validation
tests such as the Secure Coding Validation Suite.
1.7.2 False Positives
Although many guidelines list common exceptions, it is difficult if not impossible to develop a
complete list of exceptions for each guideline. Consequently, it is important that source code
complies with the intent of each guideline and that tools, to the greatest extent possible, minimize
false positives that do not violate the intent of the guideline. The degree to which tools minimize
false-positive diagnostics is a quality-of-implementation issue.
1.8 Conformance Testing
To ensure that the source code conforms to this coding standard, it is necessary to have measures
in place that check for rule violations. The most effective means of achieving this goal is to use
one or more ISO/IEC TS 17961–conforming analyzers. Where a guideline cannot be checked by a
tool, a manual review is required.
The Source Code Analysis Laboratory (SCALe) provides a means for evaluating the conformance
of software systems against this and other coding standards. CERT coding standards provide a
normative set of rules against which software systems can be evaluated. Conforming software
systems should demonstrate improvements in the safety, reliability, and security over
nonconforming systems.
The SCALe team at CERT analyzes a developer’s source code and provides a detailed report of
findings to guide the code’s repair. After the developer has addressed these findings and the
SCALe team determines that the product version conforms to the standard, CERT issues the
developer a certificate and lists the system in a registry of conforming systems.
1.8.1 Conformance
Conformance to the CERT C Coding Standard requires that the code not contain any violations of
the rules specified in this standard. If an exceptional condition is claimed, the exception must
SEI CERT C++ CODING STANDARD (2016 EDITION) | V01 12
Software Engineering Institute | Carnegie Mellon University
[DISTRIBUTION STATEMENT A] Approved for public release and unlimited distribution.