A Brief Introduction to Evidence-Centered Design CSE Report 632 ...

More documents

Recommendations

Info

ApplicationsBauer, M., Williamson, D.M., Steinberg, L.S., Mislevy, R.J., & Behrens, J.T. (April, 2001).How to create complex measurement models: A case study of principledassessment design. Paper presented at the annual meeting of the AmericanEducational Research Association, New Orleans, LA. [ECD design rationale fora simulation-based assessment of troubleshooting and design of computernetworks. Foundational analysis for the NetPASS on-line assessment ofnetworking skill, by the Cisco Learning Institute, Educational Testing Service,and the University of Maryland. Includes expert-novice analysis of problemsolving.]Cameron, C.A., Beemsterboer, P.L., Johnson, L.A., Mislevy, R.J., Steinberg, L.S., &Breyer, F.J. (1999). A cognitive task analysis for dental hygiene. Journal o fDental Education, 64, 333-351. [Expert-Novice study of expertise in problemsolvingin dental hygiene, with implications for assessment design.]Levy, R., & Mislevy, R.J. (2004). Specifying and Refining a Measurement Model for aSimulation-Based Assessment. CSE Technical Report 619. Los Angeles: TheNational Center for Research on Evaluation, Standards, Student Testing(CRESST), Center for the Study of Evaluation, UCLA.http://www.cse.ucla.edu/reports/R619.pdf. [Focus on estimation of conditionalprobability models in the Bayes net psychometric model in the Cisco LearningInstitute’s NetPASS simulation-based assessment of design, implementation,and troubleshooting in computer networks. A fairly technical psychometricpaper.]Mislevy, R.J., Almond, R.G., Dibello, L.V., Jenkins, F., Steinberg, L.S., Yan, D., &Senturk, D. (2002). Modeling conditional probabilities in complex educationalassessments. CSE Technical Report 580. Los Angeles: The National Center forResearch on Evaluation, Standards, Student Testing (CRESST), Center for theStudy of Evaluation, UCLA.http://www.cse.ucla.edu/CRESST/Reports/TR580.pdf. [Focus on estimation ofconditional probability models in the Bayes net psychometric model in theBiomass prototype assessment. A fairly technical psychometric paper.]Mislevy, R.J., & Gitomer, D.H. (1996). The role of probability-based inference in anintelligent tutoring system. User-Modeling and User-Adapted Interaction, 5,253-282. Also available as CSE Technical Report 413. Los Angeles: The NationalCenter for Research on Evaluation, Standards, Student Testing (CRESST),Center for the Study of Evaluation, UCLA.http://www.cse.ucla.edu/CRESST/Reports/TECH413.PDF.[Good foundational explanation of the use of Bayesian inference in complexassessments, illustrated with the HYDRIVE intelligent tutoring system fortroubleshooting aircraft hydraulics.]22
Mislevy, R.J., Steinberg, L.S., & Almond, R.A. (2002). Design and analysis in taskbasedlanguage assessment. Language Assessment, 19, 477-496. Also available asCSE Technical Report 579. Los Angeles: The National Center for Research onEvaluation, Standards, Student Testing (CRESST), Center for the Study ofEvaluation, UCLA. http://www.cse.ucla.edu/CRESST/Reports/TR579.pdf.[ECD perspective on designing task-based language assessments. Includesexamples of Bayes nets for tasks that tap multiple aspects or knowledge andskill.]Mislevy, R.J., Steinberg, L.S., Breyer, F.J., Almond, R.G., & Johnson, L. (1999). Acognitive task analysis, with implications for designing a simulation-basedassessment system. Computers and Human Behavior, 15, 335-374. Alsoavailable as CSE Technical Report 487. Los Angeles: The National Center forResearch on Evaluation, Standards, Student Testing (CRESST), Center for theStudy of Evaluation, UCLA.http://www.cse.ucla.edu/CRESST/Reports/TR487.pdf. [Design and conduct ofa cognitive task analysis of expertise in dental hygiene, from the perspective ofinforming the construction of the models in the ECD conceptual assessmentframework.]Mislevy, R.J., Steinberg, L.S., Breyer, F.J., Almond, R.G., & Johnson, L. (2002).Making sense of data from complex assessment. Applied Measurement inEducation, 15, 363-378. Also available as CSE Technical Report 538. Los Angeles:The National Center for Research on Evaluation, Standards, Student Testing(CRESST), Center for the Study of Evaluation, UCLA.http://www.cse.ucla.edu/CRESST/Reports/RML%20TR%20538.pdf[Argument that the way to design and analyze complex assessments, such ascomputer-based simulations, is from the perspective of the evidentiaryargument—not from the perspective of technology. Ideas are illustrated insome detail with the DISC prototype assessment of problem-solving in dentalhygiene.]Steinberg, L.S., & Gitomer, D.G. (1996). Intelligent tutoring and assessment built onan understanding of a technical problem-solving task. Instructional Science, 24,223-258. [Concerns the interplay among cognitive analysis, instructionalstrategy, and assessment design, in the context of the HYDRIVE intelligenttutoring system for troubleshooting aircraft hydraulics.]Steinberg, L.S., Mislevy, R.J., Almond, R.G., Baird, A.B., Cahallan, C., DiBello, L.V.,Senturk, D., Yan, D., Chernick, H., & Kindfield, A.C.H. (2003). Introduction tothe Biomass project: An illustration of evidence-centered assessment designand delivery capability. Also available as CSE Technical Report #609. LosAngeles: The National Center for Research on Evaluation, Standards, StudentTesting (CRESST), Center for the Study of Evaluation, UCLA.http://www.cse.ucla.edu/reports/R609.pdf Los Angeles: UCLA Center for theStudy of Evaluation. [Design rationale for a standards-based, web-delivered23
Page 1 and 2: A Brief Introduction to Evidence-Ce
Page 3 and 4: A Brief Introduction to Evidence-Ce
Page 5 and 6: models, this primer places such mod
Page 7 and 8: As powerful as it is in organizing
Page 9 and 10: ActivitySelection ProcessAdministra
Page 11 and 12: evidence in the form of behavior in
Page 13 and 14: θXjFigure 5: The measurement model
Page 15 and 16: constraints describe how tasks must
Page 17 and 18: concerns administering pre-assemble
Page 19 and 20: are all being carried out as testin
Page 21 and 22: ReferencesAlmond, R.G. Steinberg, L
Page 23: Appendix AFurther Readings about th
Page 27 and 28: Mislevy, R.J. (1994). Evidence and
Page 29 and 30: Appendix BA Glossary of Evidence-Ce
Page 31 and 32: Four Processes. Any assessment must
Page 33: Task. A Task is a unit of work requ

A Brief Introduction to Evidence-Centered Design CSE Report 632 ...

Create successful ePaper yourself

Delete template?

Save as template?