Doing It Differently: Alternative Assessment Strategies - NAGC

Doing It Differently:AlternativeAssessmentStrategiesJoyce VanTassel-BaskaJody and Layton SmithProfessor in EducationCollege of William and Mary

Table of Contents• Chapter 9: The Value of Traditional Assessments as Approaches to IdentifyingAcademically Gifted Students; Dr. Nancy Robinson• Chapter 10: Non-Traditional Applications of Traditional Testing; Dr. SylviaRimm, Dr. Barbara Gilman & Dr. Linda Silverman• Chapter 11: The Role of Creativity Tools and Measures in Assessing Potentialand Growth; Dr. Bonnie Cramond and Kyung Hee Kim• Chapter 12: Portfolio Assessment of Gifted Students; Dr. Susan Johnsen• Chapter 13: Product Assessment; Dr. Joe Renzulli and Dr. Carolyn Callahan• Chapter 14: Using Performance-Based Assessment to Document AuthenticLearning; Dr. Joyce VanTassel-Baska• Chapter 15: Epilogue: What do we know and what do we still need to knowabout identifying and assessing the learning of gifted students?; Dr. JoyceVanTassel-Baska• Appendix: Alternative Assessment Tools for Gifted Learners

Overview of Identification Issues inUsing Alternative Assessment• Debate over domain specific vs. general ability• Lack of optimal match between alternativemeasures and program models• Predictive validity of measures for success inprograms• Sufficient coverage of relevant skills in a domain

Overview of Alternative LearningAssessment Issues• Need for off-level measures to assessauthentic learning• The match between program emphasesand assessment tools• Use of assessments that tap into complexbehaviors

The Use of Nonverbal Testsfor Assessing Ability• Nonverbal approaches are as valid to get at gfactor intelligence as any other means.• School achievement is predicated on operatingeffectively in a verbal medium.• Use of nonverbal subject matter can matchbetter to students identified through suchassessment.• More low income and minority students can beaccessed for gifted programs by including anonverbal assessment tool.

Instruments in Use• UNIT Test (Bracken & McCallum)• CoGAT (Lohman & Hagen et al.)• Naglieri Nonverbal Ability Test (Naglieri)*Every test used may yield different studentsat the top. THERE IS NO MAGIC TEST.*Nonverbal tests are not a panacea forfinding underrepresented groups.

Group Discussion Activity• What nonverbal tools do you use toidentify underrepresented groups of giftedlearners?• What has been your experience with thesetools?• With what other measures do you combinethem? How?

Misconceptions about IdentifyingUnderrepresented Populations• We just need to look harder.• We need to use nonverbal measures andthey will be found.• Once we find them, then the program willtake care of the rest.• Race is a more critical consideration thanpoverty.• Higher performance on state tests is thegoal for these groups.

Factors Contributing to EducationalDisadvantagement• Education of mothers• Single parent homes• English as a second language• Poverty• Minority membership--Pallas, Natriello &McDill, 1989

African cultural components in cognitive ability testing:Hypothesized effects of African-centered values and beliefsDimension General Description Influence on Test ResponsesCommunalismExpressivenessOrality (oral traditional)Valuing of one's group(s) morethan outsiders or otherindividuals; social;interdependent.Unique personality isexpressed through one'sbehavioral styles; creative, risktaker; spontaneous.Knowledge may be gained andtransmitted orally and aurally;a preference to talk andexplain verbally.Performance may beinfluenced when test taker isanxious about the test scoresbeing reflective of his/hercultural group and havingnegative consequences forthem.Test taker may choose themore imaginative responsealternative; may be impulsive inchoosing responses.Test performance may differwhen the test taker is testedorally and aurally; test takermay be frustrated by paperpenciltest.Ford, 2008

Self Perception as a Filterto Adult AchievementSelf PerceptionEducationalAttainmentSelf PerceptionOccupationalAttainmentSelf PerceptionAdultCreativeProductivity

Assessment Tools for theTraditional•Intelligence tests•Achievement tests•Aptitude tests(domain specific)•Grades•TeacherrecommendationsIdentificationof the GiftedNontraditional•Non-verbal ability tests•Creativity tests•Student portfolios/performance by audition•Performance-basedassessment•Portfolios/creativeproducts•Parent/peer/communityrecommendations

Promising Identification Approaches withGifted At-Risk Learners• Traditional measures• Non-traditional measures• Value-added approaches (traditional plus)• Try-out activities• Nomination by educator, parent, communitymember• Use of profile rather than matrix• Consideration of risk factors in the selectionprocess

Summary of Findings From AState-wide Use of PBA• PBA protocols are consistent inidentifying more low income andminority students across 6 years ofstate-wide implementation.• PBA domain-specific protocols alsoappear to be identifying morestudents with uneven profiles thatpresent as learning problems.

Summary (cont.)• Teachers and gifted program coordinatorsview PBA-identified student strengths asbeing creative, spatial, and strong inproblem-solving while they view theirweaknesses as verbal skills, organizationskills, and time management.• PBA-identified students overall perceive theirgifted program experiences positively in allareas, with special benefits in self esteemand confidence.

Features of Performance BasedAssessment• Emphasis on thinking and problem solving, notprior learning• Off-level/advanced• Open-ended• Use of manipulatives• Emphasis on articulation of thinking processes

Sample Verbal ItemYear Round SchoolName _____________Think of all of the positive and negative effects of the following situation, andrecord them in the chart below:Situation: You have been told that your schoolwill go on a year-round schedule next year.Positive EffectNegative EffectChoose one positive effect and explain your thinking about why it would bepositive. ___________________________________________________________________________________________________________________

Sample Nonverbal ItemHalf Squares• Use the squares below to show all of the ways you can think of to shade half of thesquare. Draw more squares if you need them.4 3 2 1 04 or more pointsincluding 4 differentmethods.At least 3 pointsincluding 3 differentmethods.At least 2 pointsincluding 2 differentmethods.Only one method ofdividing is used(diagonal orhorizontal/vertical).No response.Example methods: diagonal cut, vertical cut, alternating strips, alternating quarterblocks, triangles, etc.

Student Quotes• “The program is a lot of fun. I enjoy the building, theresearching, and the computer programs… I learned tocooperate and have trust in other people, even tounderstand myself. I came to my senses—all through thegifted project in 5 th grade. Before that, I believed only inmy parents, no one else.”• “Some teachers are boring—they just tell you everythingabout the subject and then give you work to do… butthese gifted class teachers help you out with everythingand learning is not boring but fun.”• “I learned easily by seeing and hearing it, and Iremember it a lot easier than just reading it. I learned abit faster in math than other students.”

Teacher Comments• “They had a real raw curiosity, they tended not be quiteas refined as the regular kids, more innocent, a lessjaded curiosity, kind of deep curiosity to know. These areoften the kids who has some difficulty in language arts,but did extremely well in math or science or any relatedtopic.”• “Most of the Project STAR kids are from lower SESbackground, have less help at home, have lessadvantages, have been exposed to less. However, giventhe opportunity, they are very interested inlearning…they tended to have a deep level of thought,but in order to get to that, you have to probe deeper andthey have to be comfortable with you.”

Implications for the Identification ofLow Income and Minority Students• Use a mix of traditional and nontraditionaltests.• Use tests that assess both verbal andnon-verbal abilities.• Study the impact of your identificationprotocol on the inclusion ofunderrepresented groups and theirfollow-up performance in the program.

Case Studies ofUnderrepresented Groups• Sampled low income students identified asgifted through alternative measures inGrade 3 to be studied in Grade 7• Interviewed the students, their parents, agifted class teacher, and their middle schoolscience teacher• Probed their cognitive, social, and affectiveadjustment to the gifted label

Summary of FindingsLow Income African American Students(N=9)• Enjoyed the program experience• Became eager to learn• Participation itself helped self-esteem andconfidence• Diversified creative outlets• Desire for peer relationships outside oftheir gifted classes; tendency to beperceived by teachers as loners• Moral support from family

Summary of FindingsLow Income White Students (N=13)• Perceived the program positively, recognizingthe academic and affective impact of theprogram experience• Low SES did not affect their performance atschool• Both parents and teachers perceived them ascreative and had diversified talent areas• Half of the group was perceived to have learningproblems (e.g., lack of motivation, lack of timemanagement and organizational skills)• A majority of families appeared to be involved intheir children’s education

Summary of Findingsfor Twice-Exceptional Students (N=5)• Higher likelihood to be exited from the program (3 out of 5)• Both students and parents commented positively on theimpact of the gifted program• Low motivation, hypersensitivity, lack of organization skills,and negative behaviors continued to plague these students• Lack of teacher accommodations for disabilities

Cross-Prototype Themes• Students’ strengthened self-esteem,confidence, and pride in being identifiedand participating in gifted programs• Strong-willed natures• Clear preferences and strengths intargeted subject areas

Assessing Gifted Student Learning• Use of appropriate tools that exhibit technicaladequacy• Targeted, based on the intervention• Performance-based in orientation(i.e. PBA, portfolio, products)• Off-level standardized testing (e.g., SAT,EXPLORE)

Middle School Math Projects

Student math Project: MiddleSchool

Performance-based Measures• The Fowler Test (Design an experiment,based on a question)• Writing prompt (eg. Should this Book X berequired reading for your grade level?)• Literary analysis (eg. Provide a passage orshort piece and ask students to interpret)• Math (Use Math Olympiad problem sets todetermine level)

Portfolios• Ask students to select the best fiveproducts for the year to include.• Be sure they are each representative ofdifferent curricular outcomes.• Ask students to comment on why theseare their best work.• Have a portfolio exhibit.

NCATE Assessment StandardK1K2K3S1S2S3S4Processes and procedures for the identification of learners with giftsand talents.Uses, limitations, and interpretation of multiple assessments indifferent domains for identifying learners with gifts and talents,including those from diverse backgrounds.Uses and limitations of assessments documenting academic growthof individuals with gifts and talents.Use non-biased and equitable approaches for identifying individualswith gifts and talents, including those from diverse backgrounds.Use technically adequate qualitative and quantitative assessmentsfor identifying and placing of individuals with gifts and talents.Develop differentiated curriculum-based assessments for use ininstructional planning and delivery for individuals with gifts andtalents.Use alternative assessments and technologies to evaluate learning ofindividuals with gifts and talents.

Implications for School Leaders• Professional development on theidentification and assessment protocolsused in the gifted program• Longitudinal data collection on learninggains across years for all gifted groups• Focus on underperforming schools to findoverlooked gems• Provide extra support at transition points

Implications for Teachers• Move to diversify the gifted program tofocus 50% of time on nonverbal subjects(eg. math and science)• Use identification profile to plan moreindividualized approaches to instruction• Provide strong affective support foraccomplishment• Work on metacognitive skills of goalsettingand reflection

People have a basic human right to freedomfrom excessive sociocontextual obstacles toaspiration formation and self-fulfillment….deprived gifted and talented children mustembrace their considerable potentials whilerealistically appraising the daunting barriersthat stand to suppress the development ofthat potential. The onus is on educators ofthe gifted to help both the privileged and thedeprived to understand the dimensions ofself-fulfillment…-- Ambrose (2003)