Transforming Statewide High School Assessment Systems: - Achieve

Computer Adaptive Testing (CAT) takes state-of-the-art testing even further. CAT software determines what questions to ask ofan individual student based on his or her test performance during the testing session. CAT software can estimate a student’s level ofproficiency on a given test more quickly. The assessment begins with a question of medium difficulty, then offers a harder or easierquestion depending on whether the student answered the initial question correctly or incorrectly. The process continues, presentingquestions of varying levels of difficulty, until CAT can estimate with sufficient precision the student’s level of performance on theassessment. 43Some computer adaptive testing programs use test items beyond multiple choice, including constructed-response questions, morecomplicated questions that assess complex problem-solving skills and more innovative items that ask students to organize or move objectson a screen or prepare an essay.Going online with CAT has helped students in Virginia, who have to pass EOC tests for graduation, receive help in mathematics morequickly and flexibly. The state department of education worked with a computer-testing company to design the Algebra ReadinessDiagnostic Test System (ARDT) as a web-based application powered by CAT. 44 The purpose of the ARDT is to provide mathematicsintervention services to middle-level students (those in grades 6–9) who may be at risk of failing the Algebra 1 EOC test. ARDT includesa diagnostic pretest to identify student’s mathematics strengths and weaknesses, an intervention service that addresses the weaknessesthrough 2 ½ hours of additional weekly instruction and a post-test.The assessment is administered in schools on computers connected to the Internet. Results from the diagnostic test are availableimmediately and provide information correlated to the state standards and assessment reporting categories. Students targeted to take thediagnostic tests include those who were not successful in their previous intervention/remediation program, performed below average intheir previous year’s mathematics program or did not pass the state mathematics assessment.Still, Achieve notes that decisionmakers should be cautious when considering computer adaptive testing, particularly if they arethinking of using CAT as the mainstream state assessment. Because students will be tested on a wider range of questions than on apaper-and-pencil test, a sufficiently large item bank must be developed, which can be challenging and costly. 45 Access to computers,access to and speed of Internet connections, and the quality of hardware may vary across school districts, which could interfere withtest administration. State and school leaders need to be concerned about whether the existing allocations of computers to classrooms issufficient to support the amount of testing planned and whether teachers or testing proctors are trained to deal with unexpected technicalissues that may arise during testing.Transforming Statewide High School Assessment Systems: A Guide for State Policymakers 40