What do we know and what are we learning about alternate ... - NAAC
What do we know and what are we learning about alternate ... - NAAC
What do we know and what are we learning about alternate ... - NAAC
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
<strong>What</strong> <strong>do</strong> <strong>we</strong> <strong>know</strong> <strong>and</strong> <strong>what</strong> <strong>are</strong> <strong>we</strong><br />
<strong>learning</strong> <strong>about</strong> <strong>alternate</strong><br />
assessments?<br />
Jacqueline F. Kearns, Ed.D.<br />
Diane Browder, Ph.D.<br />
Pamela Mims, Ph.D.<br />
Mari Quenemoen<br />
National Alternate Assessment Center<br />
University of Kentucky<br />
www.naacpartners.org
This Session will describe<br />
the assessment population in <strong>alternate</strong><br />
assessments on <strong>alternate</strong> achievement st<strong>and</strong>ards.<br />
access to the curriculum <strong>and</strong> developing <strong>do</strong>main<br />
competence in reading, math, <strong>and</strong> science.<br />
new data on <strong>alternate</strong> assessment instruments<br />
common issues <strong>and</strong> recommendations <strong>about</strong><br />
<strong>alternate</strong> assessments.
Who <strong>are</strong> the students?<br />
Jacqui Kearns, Ed.D.<br />
University of Kentucky
Issues in Teaching/Assessing Students in<br />
Alternate Achievement St<strong>and</strong>ards Assessments<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
Varied levels of symbolic communication<br />
Attention to salient features of stimuli<br />
Memory<br />
Limited motor response repertoire<br />
Generalization<br />
Self-Regulation<br />
Meta-cognition<br />
Skill Synthesis<br />
Sensory Deficits<br />
Special Health C<strong>are</strong> Needs<br />
<br />
Kleinert, H., Browder, D., Towles-Reeves, E. (in press). Models of cognition for students<br />
with significant cognitive disabilities: Implications for assessment. Review of Educational<br />
Research.
The Assessment Triangle & Validity Evaluation<br />
Marion & Pellegrino (2006)<br />
OBSERVATION<br />
INTERPRETATION<br />
Assessment System<br />
Test Development<br />
Administration<br />
Scoring<br />
VALIDITY EVALUATION<br />
Empirical evidence<br />
Theory & logic (argument)<br />
Consequential features<br />
Reporting<br />
Alignment<br />
Item Analysis & DIF/Bias<br />
Measurement error<br />
Scaling <strong>and</strong> Equating<br />
St<strong>and</strong>ard Setting<br />
COGNITION<br />
Student Population<br />
Academic content<br />
Theory of Learning
Learner Characteristics<br />
Demographic Variables<br />
Learner Characteristics (all on a continuum of skills):<br />
Expressive Language<br />
Receptive Language<br />
Vision<br />
Hearing<br />
Motor<br />
Engagement<br />
Health Issues/Attendance<br />
Reading<br />
Mathematics<br />
Use of an Augmentative Communication System (dichotomous<br />
variable)
Metho<strong>do</strong>logy<br />
Seven partner states chose to participate<br />
during the 2006-2007 school year.<br />
States 1, 2, 3, 4, 5, <strong>and</strong> 6:<br />
gathered data in the administration process for their <strong>alternate</strong><br />
achievement st<strong>and</strong>ards assessment (i.e., bubble sheet,<br />
paper/pencil version of the LCI, etc.)<br />
State 7:<br />
gathered data using Zoomerang, an online survey package.<br />
Total N = 12,147 students
States & LCI Response Rates<br />
State<br />
Geography<br />
Participation<br />
Rates<br />
Number of<br />
Responses<br />
Number<br />
Participating in<br />
AA<br />
Response<br />
Rate<br />
State 1 North East 0.96% 2918 2918 100%<br />
State 2 Mid West 1.17% 2593 2593 100%<br />
State 3 Eastern 1.14% 3595 4768 75%<br />
State 4 North East 0.99% 722 774 93%<br />
State 5 South East 0.70% 2134 2444 87%<br />
State 6 Eastern 0.76% 468 516 91%<br />
State 7 West 0.94% 219 467 47%<br />
Totals 95.00% 12649 14480 87%
IDEA Categorical Distributions<br />
1.88%<br />
1.48% 2.94%<br />
2.00%<br />
0.75%<br />
1.00%<br />
3.30% 1.00%<br />
Mental Retardation <strong>and</strong> Multiple<br />
Disabilities<br />
Autism<br />
Other Health Impairment<br />
17.73%<br />
5.82%<br />
Autism<br />
Mental Retardation<br />
<strong>and</strong><br />
Multiple Disabilities<br />
Emotional Disability<br />
Specific Learning Disability<br />
66.85%<br />
Traumatic Brain Injury<br />
Speech Language Impairment<br />
Orthopedic Impairment<br />
Hearing Impairment<br />
Deafblind<br />
Visual Impairment
Expressive Language Definitions<br />
<br />
<br />
<br />
Uses symbolic language to communicate: Student uses verbal or<br />
written words, signs, Braille, or language-based augmentative<br />
systems to request, initiate, <strong>and</strong> respond to questions, describe<br />
things or events, <strong>and</strong> express refusal.<br />
Uses intentional communication, but not at a symbolic language<br />
level: Student uses underst<strong>and</strong>able communication through such<br />
modes as gestures, pictures, objects/textures, points, etc., to<br />
clearly express a variety of intentions.<br />
Student communicates primarily through cries, facial<br />
expressions, change in muscle tone, etc., but no clear use of<br />
objects/textures, regularized gestures, pictures, signs, etc., to<br />
communicate.
Expressive Language<br />
80.00%<br />
70.00%<br />
60.00%<br />
50.00%<br />
40.00%<br />
30.00%<br />
Symbolic<br />
Emerging Symbolic<br />
Pre Symbolic<br />
20.00%<br />
10.00%<br />
0.00%<br />
State 1 State 2a State 3 State 4 State 5 State 6 State 7
Receptive Language<br />
Independently follows 1-2 step directions presented through<br />
words (e.g. words may be spoken, signed, printed, or any<br />
combination) <strong>and</strong> <strong>do</strong>es NOT need additional cues.<br />
<br />
<br />
<br />
Requires additional cues (e.g., gestures, pictures, objects, or<br />
demonstrations/models) to follow 1-2 step directions.<br />
Alerts to sensory input from another person (auditory, visual,<br />
touch, movement) BUT requires actual physical assistance<br />
to follow simple directions.<br />
Uncertain response to sensory stimuli (e.g., sound/voice;<br />
sight/gesture; touch; movement; smell).
Receptive Language<br />
60.00%<br />
50.00%<br />
40.00%<br />
30.00%<br />
20.00%<br />
Follows Directions<br />
Requires Cues<br />
Alerts to Input<br />
Uncertain Response<br />
10.00%<br />
0.00%<br />
State 1 State 2 State 3 State 4 State 5 State 6 State 7
Use of Augmentative<br />
Communication Systems<br />
90.00%<br />
80.00%<br />
70.00%<br />
60.00%<br />
50.00%<br />
40.00%<br />
30.00%<br />
Percentage of Students<br />
NOT using AAC<br />
20.00%<br />
10.00%<br />
0.00%<br />
State<br />
1<br />
State<br />
2<br />
State<br />
3<br />
State<br />
4<br />
State<br />
5<br />
State<br />
6<br />
State<br />
7
Reading<br />
50.00%<br />
45.00%<br />
40.00%<br />
35.00%<br />
30.00%<br />
25.00%<br />
20.00%<br />
15.00%<br />
10.00%<br />
Critical Underst<strong>and</strong>ing<br />
Basic Underst<strong>and</strong>ing<br />
Reads Basic Sight Words<br />
Aw<strong>are</strong> of Text<br />
No Aw<strong>are</strong>ness<br />
5.00%<br />
0.00%<br />
State<br />
1<br />
State<br />
2<br />
State<br />
3<br />
State<br />
4<br />
State<br />
5<br />
State<br />
6<br />
State<br />
7
Mathematics<br />
60.00%<br />
50.00%<br />
Applies Computational<br />
Procedures<br />
40.00%<br />
30.00%<br />
Does Computational<br />
Procedures w/wo a<br />
Calculator<br />
1:1 Correspondence<br />
20.00%<br />
Counts by Rote to 5<br />
10.00%<br />
0.00%<br />
State<br />
1<br />
State<br />
2<br />
State<br />
3<br />
State<br />
4<br />
State<br />
5<br />
State<br />
6<br />
State<br />
7<br />
No Aw<strong>are</strong>ness
Expressive Language Across Grade B<strong>and</strong>s<br />
Elementary School<br />
Middle School<br />
12.12%<br />
19.08%<br />
Presymbolic<br />
7.58%<br />
17.03%<br />
Presymbolic<br />
65.13%<br />
Emerging<br />
Symbolic<br />
Symbolic<br />
72.57%<br />
Emerging<br />
Symbolic<br />
Symbolic<br />
High<br />
8.68%<br />
12.02%<br />
Presymbolic<br />
72.18%<br />
Emerging<br />
Symbolic<br />
Symbolic
Reading Across Grade B<strong>and</strong>s<br />
11.72% 1.78%<br />
Elementary School<br />
17.22%<br />
No Aw<strong>are</strong>ness<br />
Aw<strong>are</strong> of Text<br />
Middle School<br />
20.84% 2.75% 13.74%<br />
No Aw<strong>are</strong>ness<br />
Aw<strong>are</strong> of Text<br />
40.06%<br />
25.34%<br />
Reads Basic Sight<br />
Words<br />
Basic<br />
Underst<strong>and</strong>ing<br />
Critical<br />
Underst<strong>and</strong>ing<br />
40.34%<br />
17.82%<br />
Reads Basic Sight<br />
Words<br />
Basic<br />
Underst<strong>and</strong>ing<br />
Critical<br />
Underst<strong>and</strong>ing<br />
High School<br />
22.12%<br />
2.15%<br />
15.08%<br />
No Aw<strong>are</strong>ness<br />
Aw<strong>are</strong> of Text<br />
11.54%<br />
Reads Basic Sight<br />
Words<br />
Basic Underst<strong>and</strong>ing<br />
37.46%<br />
Critical<br />
Underst<strong>and</strong>ing
Mathematics Across Grade B<strong>and</strong>s<br />
Elementary<br />
Middle<br />
No Aw<strong>are</strong>ness<br />
No Aw<strong>are</strong>ness<br />
3.80%<br />
15.28%<br />
11.94%<br />
Counts by rote to 5<br />
1:1 Correspondence<br />
3.98%<br />
12.74% 7.98%<br />
21.98%<br />
Counts by rote to 5<br />
1:1 Correspondence<br />
34.24%<br />
30.26%<br />
Does Computational<br />
Procedures w/wo a<br />
calculator<br />
Applies Computational<br />
Procedures<br />
48.04%<br />
Does Computational<br />
Procedures w/wo a<br />
calculator<br />
Applies Computational<br />
Procedures<br />
High<br />
6.40%<br />
12.54% 4.90%<br />
No Aw<strong>are</strong>ness<br />
Counts by rote to 5<br />
17.72%<br />
1:1 Correspondence<br />
45.38%<br />
Does Computational<br />
Procedures w/wo a calculator<br />
Applies Computational<br />
Procedures
Who <strong>are</strong> the Kids in Alternate Achievement<br />
St<strong>and</strong>ards Assessments?<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
Represent ~1% or less of the total assessed population<br />
All disability categories <strong>we</strong>re represented but primarily 3 emerge,<br />
Mental Retardation<br />
Multiple Disabilities<br />
Autism<br />
Highly varied levels of expressive/receptive language use<br />
Most students in the population use symbolic communication<br />
Level of symbolic language use <strong>do</strong>es not significantly change across gradeb<strong>and</strong>s<br />
The majority of students who <strong>do</strong> not use oral speech also <strong>do</strong> not use AAC<br />
Most of the population read basic sight words <strong>and</strong> solve simple math<br />
problems with a calculator.<br />
Changes in skill progression in reading <strong>and</strong> math across grade b<strong>and</strong>s most<br />
likely due to identification of students rather than teaching <strong>and</strong> <strong>learning</strong>
Alternate Assessments<br />
Research Based<br />
Recommendations<br />
Quenemoen, Kearns, Quenemoen, Flo<strong>we</strong>rs,<br />
Kleinert (2010). Common misperceptions <strong>and</strong> research-<br />
based recommendations for <strong>alternate</strong> assessments based on<br />
<strong>alternate</strong> achievement st<strong>and</strong>ards. (Synthesis Report 73).<br />
Minneapolis, MN. NCEO
Misperception<br />
Recommendation #1<br />
Misperception… schools should not be held<br />
accountable for students who take AA-AAS… AAS…<br />
Recommendation: ensure that all students<br />
including those in AA-AAS AAS have access to <strong>and</strong><br />
learn academic content that approximates <strong>what</strong><br />
their same age peers <strong>are</strong> <strong>learning</strong> at an<br />
appropriate performance for achievement
Misperception <strong>and</strong><br />
Recommendation #2<br />
Students in AA-AAS AAS have life threatening<br />
medical conditions or <strong>are</strong> unable to<br />
communicate.<br />
All students communicate… for those students<br />
emerging in their use of symbolic language, <strong>we</strong><br />
must continue to seek communication <strong>and</strong><br />
language development strategies including<br />
assistive technology.
Meet Bruce
Misperception <strong>and</strong><br />
Recommendation Number #3<br />
Students in AA-AAS AAS should only learn rote academic<br />
skills….<br />
Arguably “rote skills” that <strong>are</strong> context free <strong>are</strong> the<br />
most difficult skills to learn & use.<br />
Our data in reading <strong>and</strong> math suggests a lack of<br />
curriculum coherence across grades…..<br />
Grade-level CONTENT is value added for social<br />
<strong>and</strong> communication skills particularly in inclusive<br />
settings<br />
Build AA-AAS AAS on a model of academic content that<br />
grade-level peers <strong>are</strong> <strong>learning</strong>
Misperception &<br />
Recommendation #4<br />
AA-AAS AAS has eliminated “functional skills”<br />
Most functional skills <strong>are</strong> taught within the context<br />
of daily routines.. Eating, toileting, safety… <strong>and</strong><br />
these should be taught in age-appropriate appropriate contexts.<br />
THE TWO MOST functional skills for life<br />
Communication<br />
Reading & literacy<br />
Provide training to teachers on merging<br />
“functional” <strong>and</strong> academic instruction….
Misperception <strong>and</strong><br />
Recommendation #5<br />
AA-AAS AAS must cover all of the same content as<br />
the general assessment.<br />
Federal regulations permit states to define the<br />
appropriate depth, breadth <strong>and</strong> complexity of the<br />
content as long as it is clearly related to grade-level<br />
expectations<br />
Provide training on research-based practices in<br />
teaching academic content to this population
Misperception <strong>and</strong><br />
Recommendation #6 & #7<br />
Most AA-AAS AAS <strong>are</strong> entirely individualized <strong>and</strong> differ for<br />
each student. Shouldn’t <strong>we</strong> just use the IEP?<br />
Current AA-AASs AASs reflect a highly varied population<br />
that when assessed together require flexibility as <strong>we</strong>ll<br />
as st<strong>and</strong>ardization.<br />
The IEP <strong>do</strong>es not provide accountability for<br />
achievement.<br />
Well designed AA-AAS AAS should utilize systematic<br />
observation procedures that allow flexibility but also<br />
control for sources of error.
Misperception <strong>and</strong><br />
Recommendation #9 & #10<br />
Some AA-AAS AAS formats (i.e., portfolio, checklist,<br />
tasks) <strong>are</strong> better than others. None <strong>are</strong><br />
technically adequate for accountability purposes.<br />
Nominal categories is not a predictor of quality<br />
Any AA-AAS AAS regardless of format can be of high or<br />
poor quality<br />
Well designed AA-AAS AAS consider the purpose<br />
<strong>and</strong> use of the assessment, the population <strong>and</strong><br />
evidence of how they learn, structured<br />
observations, <strong>and</strong> the interpretation of results
Alternate Achievement St<strong>and</strong>ards Assessments<br />
References<br />
Kearns, Towles-Reeves, Kleinert, Kleinert, & Thomas (in<br />
press). Characteristics of <strong>and</strong> implications for students participating in<br />
<strong>alternate</strong> assessments on <strong>alternate</strong> achievement st<strong>and</strong>ards. . Journal of<br />
Special Education. Retrieved www.jsped.sagepub.com<br />
Quenemoen, Kearns, Quenemoen, Flo<strong>we</strong>rs, Kleinert (2010).<br />
Common misperceptions <strong>and</strong> research-based recommendations for<br />
<strong>alternate</strong> assessment based on <strong>alternate</strong> achievement st<strong>and</strong>ards (Synthesis<br />
Report 73). Minneapolis, Mn, National Center on Educational<br />
Outcomes.
Contact Information<br />
Jacqueline Kearns, Ed.D.<br />
1 Quality Street, Suite 722<br />
Lexington, Kentucky 40507<br />
859-257<br />
257-7672<br />
7672<br />
859-323-1838<br />
<br />
Jacqueline.kearns@uky.edu<br />
<br />
Mari Quenmoen<br />
http://www.naacpartners.org/
Change language