MAKING VALID AND RELIABLE DECISIONS IN ... - CCSSO projects

Reliability of Assessment Results: The degree to which the scores of every individual are consistent overrepeated applications of a measurement procedure and, hence, are dependable and repeatable; the degreeto which scores are free of errors of measurement.Sample: A sample is a selection of a specified number of entities called sampling units (test takers, items,etc.) from a larger specified set of possible entities, called the population.Sampling: The selection of a sample.School Report Cards: Reports that provide information about schools, as a whole, rather than aboutindividual students. For example, they may include information about the number of students who scoreat the proficient level on State tests, information about the number of teachers teaching in their areas ofprimary training, as well as information about attendance, retention, and discipline referrals. In somecases, the data on school report cards are used to make programmatic decisions about schools or todetermine whether they meet accreditation criteria, for example.Secure Forms of Assessments: Refers to the need to keep high-stakes tests safeguarded so that all studentshave equal exposure to the test materials and equal opportunities for success. If test security is violated,then some students can be placed at an unfair advantage or disadvantage. When this happens, thevalidity of high-stakes tests is violated.Stakeholders: Persons holding a vested interest in the outcomes of the assessment program. These likelyinclude parents, students, educators, and taxpayers.Standard Assessment: Refers to the administration of an assessment in the prescribed, standard way,without the use of accommodations or modifications.Standard Error of Measurement: The average amount that scores in a distribution differ from thecorresponding true scores for a specified group of test takers.Standards-Based Tests: A type of criterion-referenced test. They consist of items that reflect a preestablishedset of content standards. Results are then interpreted against a set of criteria orperformance standards.Technically Sound Accountability Systems: Systems that are defensible, reliable, and valid for thepurposes for which they are used, fair, and unbiased.Test Forms: Parallel or alternate versions of a test that are considered interchangeable, in that they measurethe same constructs, are intended for the same purposes, and are administered using the same directions.True Scores: In classical test theory, the average of the scores that would be earned by an individual on anunlimited number of perfectly parallel forms of the same test. In item response theory, the error-freevalue of test taker proficiency.Valid: Refers to the degree to which a test measures what it purports to measure. See Validity.Validity of a Test:(1) An overall evaluation of the degree to which accumulated evidence and theory support specificinterpretations of test scores(2) The extent to which a test measures what its authors or users claim it measures(3) The appropriateness of the inferences that can be made on the basis of test resultsValidity of the Accountability System: An accountability system can be said to have validity when theevidence is judged to be strong enough to support the inferences that:• The components of the system are aligned to the purposes, and are working in harmony tohelp the system accomplish those purposes; and• The system is accomplishing what was intended (and did not accomplish what was not intended.)104 Making Valid and Reliable Decisions in Determining AYP