Desk Monitoring Guide - National Reporting System

they exit to collect data and achieve this level of response. In addition, the NRS employmentmeasures are time bound: entered employment must be collected after the first quarter after exit andretained employment after the third post-exit quarter. States relying on data matching are sparedhaving to conduct a survey but must collect students’ Social Security numbers. These requirementssuggest two measures for desk monitoring.• Number of students contacted by time. Data on how many students the program isreaching and whether it is contacting the students regularly and within the correct timeframe allow desk monitors to know whether the survey is on track toward reachingstudents and achieving a good response rate. Programs could report this number in totalor broken down by each of the four follow-up measures.• Number of students with Social Security numbers. The biggest obstacle to collectingfollow-up measures through data matching is missing students’ Social Security numbers.Regularly monitoring the proportion of Social Security numbers programs have collectedallows staff to identify and correct potential problems before it is too late.Keeping track of survey response rates, when programs conduct their surveys, and whetherprograms are collecting Social Security numbers will minimize missing data and improve the validityof follow-up measures.Program Process MeasuresProgram process measures provide indicators of the services that programs provide and howthe program works—the procedures and services performed that affect outcomes. Although outcomemeasures tell us what students are achieving and data process measures provide indicators ofprocedures affecting data collection, program process measures indicate what actually goes on in theprogram to produce the outcomes. Accountability systems like the NRS assume that good outcomesresult from good programs. Including measures of program processes in desk monitoring allows youto verify this assumption. These measures also help you understand why a program is performing theway it is and offer clues for identifying reasons for good and poor performance to guide programimprovement efforts.Most states have general indicators of good program performance that they use in monitoringand evaluating programs. In the early 1990s, the National Literacy Act (NLA) required states todevelop model indicators of program quality for adult education. Using the model indicatorsdeveloped by OVAE, most states developed quality indicators to define good program policy andpractice in the following areas:• Recruitment and enrollment• Retention and attendance• Professional development and staff• Curriculum and instruction• Support servicesMany states continue to use these indicators to guide their program monitoring and programimprovement efforts.A thorough examination of program process based on quality indicators requires more directobservation and contact with programs through on-site monitoring. However, you can include in a12 Desk Monitoring: Improving Program Performance