CMT-CAPT Skills Checklist Technical Manual. - NAAC

More documents

Recommendations

Info

ReliabilityReliability questions relate to the extent to which an assessment instrument producesconsistent results over time. With an instrument such as the CMT/CAPT Skills Checklistwhich requires teachers to rate student performance in different settings over a period oftime, a critical consideration is the extent to which raters apply scoring criteria similarlyand consistently over time. As was described previously (See “Preparing Raters,” Page28) extensive ongoing training has been provided to teachers who use the CMT/CAPTSkills Checklist to ensure consistent administration of the instrument. Discussions arecurrently underway to determine how best to supplement planned 2007-2008 trainingactivities, including training to criterion, with a study to determine inner-rater reliabilitycoefficients for individuals who are using the Checklist.Internal ConsistencyInternal consistency (alpha) reliabilities were computed for all components of thechecklist at each grade level. These are shown on the following tables. The reliabilityindices are uniformly high for the Access Skills as well as for the three content areasassessed; Reading, Communications and Mathematics. For the total population, thereliability at each grade ranged from .96 to .99 for each subtest. Analyses for thedisaggregated student population based on race, gender and Free Lunch status yield allreliability estimates to equal or exceed .93 with most of the estimates in the range from.97 to 1.0. Although the CSDE does not report scores from the checklist at the strandlevel, strand level reliability estimates are provided here as they are used for further testdevelopment purposes. These estimates were also generally high at .90 or higher, withmost being from .94 to .99. In only three cases (Basic Literacy in grade 5 and ExpressiveCommunication and Basic Spatial Relations in grade 6) did the reliability estimate dropbelow .90. These areas will be reviewed during the next phase of test development.These results, as presented in Tables 4 through 6, suggest that the checklist has beenapplied in a highly consistent manner in each of the grades assessed."14
Table 4Internal consistency (alpha) reliabilitiesBy Subgroup by GradeWithout ExclusionsSubgroupGrade Level3 4 5 6 7 8 10All 0.99 0.99 0.99 0.99 0.99 1.00 0.99American Indian * * * * * * *Asian American * * * * * * *Black 0.99 0.99 0.99 0.99 0.99 0.99 0.99White 0.99 0.99 0.99 0.99 0.99 0.99 0.99Hispanic 0.99 0.99 0.99 0.99 0.99 1.00 0.99Male 0.99 0.99 0.99 0.99 0.99 1.00 0.99Female 0.99 0.99 0.99 0.99 0.99 0.99 0.99Free lunch 0.99 0.99 0.99 0.99 0.99 1.00 0.99* = n size < 4015
Page 1 and 2: DRAFT: October 19, 2006CONNECTICUT
Page 3 and 4: CMT/CAPT SKILLS CHECKLISTTECHNICAL
Page 5 and 6: SECTION I—OVERVIEW, BACKGROUND, A
Page 7 and 8: In addition, the purpose of Connect
Page 9 and 10: Thus, we - the public as well as ed
Page 11 and 12: assessment should provide similar i
Page 13 and 14: The Second Generation CMT/CAPT Skil
Page 15 and 16: Description of students and alterna
Page 17 and 18: from a variety of contexts or to co
Page 19 and 20: illustrates how to link all of the
Page 21 and 22: Table 2TOTAL DOWNWARD EXTENSIONS BY
Page 23 and 24: 3. What contributes to understandin
Page 25 and 26: Orientation meetingIn early June 20
Page 27 and 28: Guided item development processIn t
Page 29 and 30: Who administers the Checklist?The C
Page 31 and 32: • Implications for the inclusion
Page 33 and 34: • a CD with the materials utilize
Page 35 and 36: Video recordings selected to serve
Page 37 and 38: Completing the CMT Skills Checklist
Page 39 and 40: Completing the CMT Skills Checklist
Page 41 and 42: The specific attestation in Section
Page 43 and 44: Database audit trackingAll student
Page 45 and 46: (A&EC), along with its sister compa
Page 47 and 48: Appendix C) approved the standard s
Page 49 and 50: identifies the student’s performa
Page 51: match between the test items for su
Page 55 and 56: Table 6Internal consistency (alpha)
Page 57 and 58: APPENDIX ACMT/CAPT Skills Checklist
Page 59 and 60: Patricia MoranCharlene Tate Nichols
Page 61 and 62: CMT/CAPT Alternate Assessment Skill
Page 63 and 64: Bill WalkerBeth WenzelJoanne WhiteC
Page 65 and 66: Technical Advisory CommitteePeter B
Page 67 and 68: Table 1ITEM BY STRAND (STANDARD): M
Page 69 and 70: Table 3TOTAL POSSIBLE POINTS: MATH
Page 71 and 72: Table 5TOTAL DOWNWARD EXTENSIONS: L
Page 73 and 74: APPENDIX EExamples of Downward Exte
Page 75 and 76: B. Students interpret, analyze and
Page 77 and 78: PARAMETERS OF ITEMS (I.E., DOWNWARD
Page 79 and 80: APPENDIX GStandard Terminology41
Page 81 and 82: APPENDIX HCMT/CAPT Skills Checklist
Page 83 and 84: APPENDIX JAlignment Analysis of Con
Page 85 and 86: TABLE OF CONTENTSIntroduction......
Page 87 and 88: • Pamela Brucker, Southern Connec
Page 89 and 90: Mathematics Framework were reformat
Page 91 and 92: corresponding to a standard had to
Page 93 and 94: • Use punctuation marks correctly
Page 95 and 96: Other Level 3 activities include dr
Page 97 and 98: Objective 2.1.a.Construct polygons
Page 99 and 100: The alignment analysis for grade fi
Page 101 and 102: The alignment analysis for grade te
Page 103 and 104:
The alignment analysis for grade fo
Page 105 and 106:
The alignment analysis for grade ei
Page 107 and 108:
ReferencesValencia, S. W., & Wixson
Page 109 and 110:
Table 1`Language Arts: Reading and
Page 111 and 112:
Table 3Standards by Performance by
Page 113 and 114:
Table 5Connecticut Alternate Assess
Page 115 and 116:
APPENDIX MLearner Characteristics I
Page 117 and 118:
APPENDIX ORelating Items from the C
Page 119 and 120:
9/10-1 Activate prior knowledge, es
Page 121 and 122:
APPENDIX PScope and Sequence Tables
show all

CMT-CAPT Skills Checklist Technical Manual. - NAAC

Create successful ePaper yourself

Delete template?

Save as template?