MAKING VALID AND RELIABLE DECISIONS IN ... - CCSSO projects

making AYP will be included in the Secretary of Education’s annual report “to the Committee onEducation and the Workforce of the House of Representatives and the Committee on Health,Education, Labor, and Pensions of the Senate…” (Section 6164).The 2001 ESEA Reauthorization continues the national emphasis on standards, annualassessments of student learning, accountability for results, and school improvement. TheReauthorization adds new requirements for the annual assessment of the acquisition of Englishlanguage proficiency by limited English proficient students and mandatory requirements forschool districts (receiving Title I funds) and their schools, if selected, to participate in the NationalAssessment of Educational Progress (NAEP).Accountability—A Change inDirection andEmphasis.However, the No Child Left Behind Act of 2001 has most surely andmarkedly changed the direction of educational accountability for thenation’s public school districts, public schools, and States. Under the 1994ESEA Reauthorization, all parties—schools, districts, and States—wereheld accountable for annual, measurable increases (i.e., progress) towardthe goal of having all students achieve to high academic standards (which could be based on acompensatory model to determine whether the required amount of progress was being made).A compensatory model or approach allows higher scores on some measures to offset (i.e.,compensate for) lower scores on other measures. For example, higher performance in languagearts could be used to offset lower performance in mathematics. The most common example of thecompensatory approach is the simple average. There are many more complex compensatorymethods, such as many of the standards-setting processes used in large-scale assessment programs,but for now, thinking about the compensatory approach as a simple average will suffice.Consistent with the 1994 Reauthorization then, States began to set their own timelines and AYPrequirements for the attainment of this goal. Although performance data were required to bereported publicly by disaggregated groups, AYP determinations were made on the basis of theperformance of all students and not on the basis of the performance of subgroups of students.In the new accountability system under the NCLB Act, adequate yearly progress requirements areprecisely set in the law and each student subgroup is required to meet or exceed its annualmeasurable objective each year—a measure of status; requiring that each of the various subgroupsof students be at or above the performance targets for a given school year (which means that only aconjunctive model can be employed). The NCLB Act holds all parties—schools, districts, andStates—accountable for helping all students, including specific subgroups of students, to achieveto each State’s proficient or advanced levels in reading or language arts and mathematics within 12years beginning with the 2002-03 school year. (There are three proficiency levels specified in thelaw—advanced, proficient, and basic—although they are not specifically defined in the law or byregulations.)In a conjunctive model or approach, scores on all measures used must be above the criterionpoint (cut score) for the student to have met the overall standard. If three measures were used todetermine whether or not a student has met a standard, the student would have to be above the cutscore on each measure to be considered proficient. This is a fairly stringent approach and typicallyleads to the lowest pass rate.The NCLB Act, contrary to the 1994 Reauthorization, also sets a minimum assessmentparticipation rate applicable to all students and to each subgroup of students specified in the law.The Act further requires States to include at least one other academic indicator (in addition to theState assessment) of student performance in the determination of AYP. In this regard,8 Making Valid and Reliable Decisions in Determining AYP