CMT-CAPT Skills Checklist Technical Manual. - NAAC

chosen by AEC/BETA and were external to the state. The major deliverable for thisalignment study was a clear and readable report delineating the results of the study.Throughout the alignment institute, reviewers concentrated on the four criteria central tothe Webb Alignment method:• Categorical Concurrence – the criterion of categorical concurrence betweenstandards and assessment is met if the same or consistent categories of contentappear in both standards and assessment documents;• Depth-of Knowledge (DOK) consistency – Depth-of-knowledge consistencybetween standards and assessment indicates alignment if what is elicited fromstudents on the assessment is as demanding cognitively as what students areexpected to know and do as stated in the standards;• Range-of-Knowledge (ROK) correspondence – The range-of-knowledgecorrespondence criterion is used to judge whether a comparable span ofknowledge expected of students by a standard is the same as, or corresponds to,the span of knowledge that students need in order to correctly answer theassessment items/activates; and• Balance of Representation – The balance-of-representation criterion is used toindicate the degree to which one objective is given more emphasis on theassessment than another.ResultsThe results of this alignment study of the CMT/CAPT Skills Checklist determined that“for the alternate assessment, the content standards and the assessment items are verywell aligned with respect to all four alignment criteria – categorical concurrence, depthof-knowledge,range-of-knowledge, and balance of representation.” (Alignment Analysisof Connecticut’s Standards and Assessments: Executive Summary, Assessment andEvaluation Concepts Inc., April 2006, Page 18) A copy of this report is included inAppendix J.STANDARD SETTINGStandard setting methodologyThe standards for the CMT/CAPT Skills Checklist were set in January 2006, prior totesting because of a deadline imposed on the CSDE by one of the testing contractors.The Checklists were to be administered for the first time in March 2006. Unlike thestandard CMT and CAPT where there are five achievement levels (below basic, basic,proficient, goal and advanced), on the Skills Checklist there are only three achievementlevels (basic, proficient, independent). Thus, it was necessary to establish performancestandards (cut scores) for the achievement levels on the Checklist prior to theiroperational use. In their January meeting, the Technical Advisory Committee (see8