Anonymous (XXXX) Rubric scoring and item writing.pdf

More documents

Recommendations

Info

Alternative Assessments Example Benefit Limitation Individual Projects Require creativity, originality & Be sure each employee has the integration ofmany skills same experience or access to resources Group projects Assess ability to work with others Groups have an impact on individual behavior, can't assess what an individual has done or achieved Portfolio Student participation Scoring rubric has to be carefully "best work" constructed "growth" . Demonstration Less complex than a project Need clear learning target & scoring rubric Experiment Can assess whether students use Time consuming---expensive in proper inquiry skills & methods materials "process" Oral Presentation Assess skills not able to be Must decide how to weigh Debate assessed with test "project" content vs. performance(well Dramatization presented, but material was all wrong) Structured Tasks Can be linked to teaching Scoring rubric (write story, build something activities Naturally occurring tasks You have to wait for the event to occur • Extended response essay item is particularly useful to assess student ability to organize ideas and include relevant infonnation--disadvantage is that scoring is inconsistent- can be problematic for elementary school students • Restricted response items can be used for more than just rec~ll and comprehension, but they need to be carefully crafted-advantage-it gives more reliable results (responses will be similar because response fonnat is focused and organized in similar way for all students) • Best item formats: • Matching-symptoms & contaminants • Recall facts-short-answer (don't use PA for this) • Comprehension/basic understanding-mc • Rater drift---results in inconsistent scoring • Carry-over effeets-to minimize grade all #1 items ofall students, then go on to next item & do for all students • Rubrics are designed to improve reliability ofessay scores and PA -they define different levels of quality the instructor will use to evaluate essay responses or performance • Can use verbal descriptions ofquality at each level • Can use examples ofprevious students performance or products • Analytic Scoring rubrie-gives more detailed feedback-helps students know what to expect, so they can assess their own work before turning it in-helps students identify areas which need remediation • Holistic scoring rubric-less objective • Process/procedure vs. product • Problem solving vs. story or display • PA-unstructured vs. structured-scaffolding-directions or structure provided to students---less structure means less "scaffolding", i.e., linking ofsuccessive task requests • Checklist-can cover positive and negative behaviors .. • Portfolio-purpose is important-what will results be used for-promotion to next grade?
Thinking Skills, Trigger Words, and Evaluative Criteria _ adapted by W. D. Schafer from the Quellmalz & Bloom et al. taxonomies and Stiggins's materials Quellmalz proposes five categories (QI-Q5, below) but the order is·not viewed as important. Bloom et a/. propose six cat~gories (BI-B6, below) ordered by complexity of the cognitive operation. Recall (Q1) - verbatim repetition [knowledge (Bl)] and paraphrasing [comprehension(B2)] Trigger Words (knowledge): define, label, locate, recite, name, state Trigger Words (comprehension): describe, restate, paraphrase, rewrite, express, identify Evaluative Criteria: accuracy, comprehensiveness Analysis (Q2 & B4) - division into component elements (e.g., part-whole, cause-effect) Trigger Words: analyze, subdivide, outline, order, inventory Evaluative Criteria: identification (sufficiency & importance), description, explanation, justification Comparison [Q3 (& B4)] - identification of similarities and differences Trigger Words: compare, contrast, categorize, sort, separate, classify, differentiate Evaluative Criteria: distinctiveness, sufficiency, accuracy, explanation, clarity Inference (Q4) - deductive reasoning [application (B3)] and inductive reasoning [synthesis (B5)] Trigger Words (application): predict, demonstrate, apply, solve, use, sketch Evaluative Criteria: choice of rule (generalization), plausibility of argument, accuracy of result Trigger Words (synthesis): construct, formulate, speculate, hypothesize, theorize Evaluative Criteria: use of data, identifying generalization, logic of argument, noting of exceptions Evaluation [Q5 & B6 (& B4)] - judging worth, quality, credibility, or practicality Trigger Words: prioritize, judge, evaluate, appraise, recommend, critique, weigh, [question] & why Evaluative Criteria: data orientation, explanation of criteria, defensibility of application of criteria (note - it is not the judgment, but its explanation and the argument presented that is rated) From Fleming & Chambers (1983): Percent of Actual Teachers' Items by Cognitive Levels (n=8800 items) Cognitive Level (revised from their use of the Bloom et a/. categories) Level in School Recall Analysis Comparison Inference Evaluation Elementary 83% 10% 0% 7% 0% Junior High 97% 3% 0% 0% 0% High School 97% 3% 0% 0% 0% (note: most inference items are found in mathematics and are deduction, or application of principles) Sample table of specifications [%'s are emphases for objective (an operation using a content) classes]: Content Area (Civil War) During War Post-War 25% 5% 5% 5% Operation (Cog. Level) Recall Analysis Comparison Inference Evaluation Total Pre-War 5% 5% 5% 5% 5% 25% 10% 10% 50% 5% 10% 25% Total 35% r 15% 15% 20% 15% 100% Avoid the "recall trap," where test items seem to measure higher-order thinking but the specific instances have been taught in class, so the students are merely remembering material as it has been taught. To engage higher-order thinking, the student must be presented with novel material. Beyond thinking skills are thinking processes, when skills are used in sequence or in combination. Examples are problem solving, decision making, and scientific inquiry.
Page 1 and 2: When Designing and Developing a Per
Page 3 and 4: General Benefits General Limitation
Page 5 and 6: 2 Example ofRubric for Math problem
Page 7: Performance Assessment Advantages &
Page 11 and 12: Robe~t w. Lissitz First compiled by
Page 13 and 14: 33. Arranqe the options in some log
Page 15 and 16: 11. Make gross judgments of the rel
Page 17 and 18: -2- Lissitz 11. World War II was fo
Page 19 and 20: DIREcrIONS: Column A. belOil consis
Page 21 and 22: DIRECI'IONS: Match each of the poet
Page 23: DIRECI'IONS: In each of the columns

Anonymous (XXXX) Rubric scoring and item writing.pdf

Create successful ePaper yourself

Delete template?

Save as template?