Code Manual for CONTAIN 2.0 - Federation of American Scientists

In addition to simpler calculations traditionally done at this testing stage, developmental testing also
includes actual plant calculations. Including more complex calculations at this phase of testing
exposes errors not previously discovered. Also, the calculations help in finding any unexpected
results related to model interdependencies. It should be noted, however, that developmental testing
cannot be very comprehensive, especially for code updates that make changes in many areas of the
code. Therefore, the Change Document must indicate what developmental testing has been
performed, to identify the aspects of the updates have been checked or verified.
When the Change Document for the update is complete and approved, the update is frozen and a
transition is made from developmental testing to internal beta testing. In the developmental testing
stage, the developer is allowed to change the update to correct any problems discovered. In contrast,
in the internal beta testing program the update is frozen and direct modifications to the update are
not allowed. Any changes to the update that are subsequently found to be desirable must be
implemented through subsequent updates.
D.2.6.2 Internal Beta Testing. After approval of the Change Document for an update, testing can
proceed to the internal beta testing stage. In the internal beta testing phase, the purpose is to exercise
the update to uncover problems or errors not detected in the developmental testing stages. No formal
documentation is generated as part of the internal beta testing. Internal beta testing is typically
performed by a member of the CONTAIN staff other than the developer. Experienced users outside
of the CONTAIN staff may also perform internal beta testing.
Internal beta testing replaces independent testing, which was the prescribed practice prior to
CONTAIN 1.12. Independent tests were performed for the earlier versions of CONTAIN and as a =
result, CONTAIN had reached a demonstrated level of robustness and maturity. Consequently,
independent testing has become outmoded and internal beta testing is considered more appropriate
and cost-effective means of verifying revisions and new models implemented in CONTAIN.
D.2.6.3 Jntemated Testing. When the internal beta testing is successfully completed and the Change
Document has been approved, the revision manager includes the update in the revision. At this
stage, the update has been tested against the established code and against all updates accepted ahead
of it in the revision. Before the final revision is released, all updates are tested together using the
complete set of standard tests, plus selected plant calculations deemed appropriate by the revision
manager. This process is defined as integrated testing. This integration analysis verifies the ability
of the collected updates to perform accurately and consistently as part of an integrated code. This
pattern of testing causes several tests to be applied more than once—a redundancy that is necessary
for a good quality assurance program.
The present set of standard tests (see Table D-2) provides a measure of quality assurance for the
compatibility of all the updates. Jn addition, an automated process has been developed to perform
a quantitative check of differences between CONTAIN results performed on different computer
systems (SUN, HP, DEC, and IBM machines) to help identify and guard against machine-dependent
coding. In this automated process, standard tests are run on the different machines and the results
are then compared.
Rev. O D-18 6/30/97