Desk Monitoring Guide - National Reporting System

should be as simple as possible so that local programs could easily understand and use it. Acommittee that included three local practitioners (a teacher and two directors) then worked out thedetails of designing the system and then presented it to the field for input.Kansas uses an advisory committee to help guide its monitoring approach. Each year, thecommittee reviews the measures, including the indicators of program quality, in the desk monitoringand suggests revised measures that are based on past experience and new policy priorities. The statenotifies programs of changes a year in advance to give them a chance to adjust their goals andprocedures.After a few years of working on its desk monitoring approach, Ohio formed an accountabilitytask force to guide the approach. This group, which meets quarterly, includes state and fieldrepresentatives who provide input on key issues related to the NRS and state accountability issues.Over the last year, the task force had to approve all revisions to the desk monitoring process. Thetask force advised the state of the need to inform the field about changes prior to the beginning of theyear of implementation of the new requirements. It also advised the state to roll out new measuresfirst as bonus points and then in the following year to integrate changes into the scoring system.Texas spent several months with different activities to build in support and understanding ofits monitoring tools, including local program report cards, which the state includes in its deskmonitoring. For example, the state held a three-day retreat with local program directors, who had toenter their own data and interpret and explain their program outcomes to their peers, using themonitoring tools. The directors then had to train their local staff on using and interpreting the data.This experience helped them understand and support the process and the state approach towardprogram improvement.New York’s experience implementing its local program report cards (which also serves as adesk monitoring tool) highlights the importance of collaborating with local programs. In the haste toget the report cards implemented, the state initially had little collaboration with the field.Consequently there was some resistance, with many programs feeling blindsided by the process. Insubsequent changes and enhancements to the report cards, New York has had greater collaborationwith better results and more acceptance.Field Test and Train StaffAlthough input from the field and other stakeholders is important to good planning, pilottesting of the system is essential to ensure that your state’s approach will work. The desk monitoringapproach of using data may be new to your local programs. Several states emphasize the need to testdifferent versions of the monitoring tool with different measures and formats to get local programsused to the process. Pilot testing gives local programs the opportunity to gauge the impact on theirprocedures, allows the programs to see how the state will evaluate them, and gives them theopportunity to make changes to implement the new reporting requirements that desk monitoringbrings. Pilot testing is also another opportunity for getting local buy-in to the process. New York, forexample, had local programs try procedures for the first year with a limited set of measures and withno consequences before the state fully implemented its approach.Distributing to local programs draft versions of your desk monitoring tool, procedures, andreports is another way to evaluate your approach before full implementation. Draft report templatesthat include blank data tables or simulated data will allow local staff to see the measures planned fordesk monitoring. Including sample reports with graphs and charts to illustrate the way the state willDesk Monitoring: Improving Program Performance 29