PDF, 3.8 MB - Substance Abuse and Mental Health Services ...

capable of doing the work would receive the necessary help both during and after training to
interview successfully on the NSDUH.
Through the certification process (see Section 5.2.1), formal one-on-one evaluation of
each trainee occurred. As explained earlier, all trainees were required to complete the
certification in order to successfully complete training.
In addition, all new-to-project graduates were mentored (see Section 5.2.5) to observe
their behavior in the field and reinforce the important study protocols learned during training.
8.1.3 Observations at Veteran Training/Ongoing FI Knowledge Evaluations
Veteran FIs continuing work on the study in 2003 were tested and trained to be sure they
met the standards necessary to serve as NSDUH interviewers. Beginning with the electronic
home study (see Section 4.5.1), interviewers could only continue working if they demonstrated
knowledge of basic protocols. During veteran training, FIs were monitored through classroom
performance.
Periodic evaluations (eVals) of interviewer knowledge occurred during the year (see
Section 5.5). This tool not only tested knowledge but reinforced that following protocol helps
collect data of the highest possible quality. All interviewers also received a laminated copy of the
form "Steps to Maximize Data Quality" (see Exhibit 8.1) which listed the most crucial NSDUH
protocol steps.
8.1.4 Field Interviewer Observations
In-person observations of FIs at work provided insights about the survey and its
procedures as well as assessments of interviewer performance and attention to project protocol.
Field Observations were implemented nationally for the second, third, and fourth quarters of
2003.
Around the country, 319 FIs were observed completing 638 screenings and 414
interviews. Observers, who were regional directors (RDs), RSs, FSs, members of the
Instrumentation Team, project survey specialists, or Substance Abuse and Mental Health
Services Administration (SAMHSA) staff, had specific forms to complete, noting interviewer
behaviors on a number of project protocols. Data from completed forms were used to assess
current levels of interviewer knowledge and develop training plans to improve FI skills in
identified problem areas. To maintain the integrity of the operation, observers did not give direct
feedback to the FIs. Information regarding FI performance was made available to the appropriate
FS to share with observed FIs. Results from these observations were formally documented in the
2003 NSDUH Full-Year Field Observation Report.
8.1.5 FS Quarterly Evaluations of FIs
At the end of every quarter of data collection, each FS evaluated the FIs in his/her region
to decide how to allocate bonus funds and whether to recommend any merit-based pay raises.
FSs considered all the facets of being a "good FI," including production, response rates,
adherence to procedures, costs, timeliness, attitude, commitment, attention to details, lack of data
332