Standard CMMI Appraisal Method for Process Improvement (SCAMPI)

1.4.2 Administer Instruments (continued)
Optional
Practices
Implementation
Guidance
Establish an organizational asset (or rely on an existing one) that documents
and maintains the traceability of implemented practices to model practices.
Conduct a workshop to document the PIIs for the organization.
The use of instruments to gather written information from members of the
organization provides a relatively low-cost data collection technique when
done well. Data of this type tend to be most useful when provided early in the
appraisal conduct, and can lead to valuable insights about where data may be
sought during subsequent data collection events. Since there is limited
opportunity for elaboration and “branching” to related topics, responses to
instruments can sometimes raise more questions than they answer for the
appraisal team member trying to interpret the responses. Furthermore,
instruments that contain excessive jargon or complicated terminology may
hinder data collection rather than help. Confused respondents will do their
best to answer the question they don’t quite understand, and the response is
interpreted based on the question that was intended. Having a knowledgeable
person present during the administration of an instrument can help mitigate
the risk of miscommunication.
One of the attractive features of instruments for the purpose of data collection
is that they can be used to establish a “scoring scheme” that reduces the
burden of interpretation for the recipient of the data. Such schemes do not
exist for SCAMPI, and the use of a shortcut of this type is a violation of the
principle that focuses rating judgments on the goals of the PAs in CMMI
models. The practices found in CMMI models are Expected Components,
while the goals in the models are Required Components. While the
satisfaction of a PA goal is predicated on the implementation of practices
found in the model (or acceptable alternatives), there is no strict aggregation
scheme that allows one to infer goal satisfaction based on practice
implementation alone. Rating judgments are based on multiple sources of
objective evidence and the reasoned consideration of strengths and
weaknesses, in aggregate.
Whenever possible, documents mentioned in the responses to questionnaires
or other instruments should be requested for team review early in the process,
so that any misleading references will not cause undue confusion later.
CMU/SEI-2001-HB-001
Page II-53