Source Code Analysis Laboratory (SCALe) for Energy ... - CERT

3 Conformance Testing
3.1 Introduction
In general, objective third-party evaluation of a product provides confidence and assurance that
the product conforms to a specific standard. The CERT SCALe assesses a software system, determines
if it conforms to a CERT secure coding standard, and provides evidence to that effect.
The services are performed under a service agreement.
Conformance testing by a recognized and respected organization such as CERT ensures the impartiality
of the assessment, ensures fair and valid testing processes, and fosters confidence and
acceptance of the software by consumers in the public and private sectors.
According to the results of a recent survey conducted for the Independent Association of Accredited
Registrars (IAAR), the main motives organizations cited for obtaining a third-party certification
of conformance to a quality standard were “customer mandate” (29 percent), “competitive
pressure or advantage” (17 percent), “continuous improvement based on customer requirements”
(16 percent), and “improve quality” (14 percent). Less frequently cited were “implementation and
control of best practice” (10 percent) and “corporate mandate” (9 percent). “Reduce cost,” “risk
management,” and “legal reasons” were each cited by 1 percent of respondents [ANAB 2008].
For many organizations, product certification yields financial benefits because of cost reduction
and new sources of revenue. Among respondents to the IAAR survey, 86 percent of companies
certified in quality management realized a positive return on investment (ROI). An ROI of more
than 10 percent was reported by 26 percent of respondents to the survey.
While undergoing third-party audits to become certified may be voluntary, for many organizations
there are compelling reasons to do so:
• improve the efficiency and effectiveness of operations
• satisfy customer requirement
• satisfy contractual, regulatory, or market requirement
• instill organizational discipline
• demonstrate to shareholders, regulators, and the public that a software product has been audited
• instill customer confidence
• identify issues that may be overlooked by those inside the organization, providing fresh internal
improvement strategies
Common elements of conformance assessment include impartiality, confidentiality, complaints
and appeals, and information disclosure policy.
CMU/SEI-2010-TR-021 | 28