Presentation Handout - Center on Teaching and Learning

“Assessment and Use of Data toDrive Improvement Efforts”CTL’s Research to Practice Conference:Professional Development for the21 st CenturyLeadership for ImprovingLiteracy Outcomes at the Schooland District Level StrandOctober 29 & 30, 2012Sandra D. Jones, Ph.D., HILL for Literacy, Inc.sandy@hillforliteracy.orgwww.hillforliteracy.orgGoals Today1. Develop an understanding of aleader’s role in assessing andgrouping students based on data.2. Link leadership routines for makingdata-based decisions to improveinstruction of struggling students toprofessional development.3. Review protocols that ease theprocess of making data-baseddecisions during a school-wideliteracy change initiative.www.hillforliteracy.org 1

Component 1OutcomesAssessmentCritical Inquiry• DynamicAssessment• Data Analysis• FlexibleGrouping• InstructionalGoals & Practice• Data-basedLeadershipFocus of this component is ondata-based instruction and theleadership decisions that leadto improved student outcomes.Key Deliverables• Summative &Formative Data• Analysis of Data• Strengths/Weaknessesof Implementation• Strategic Leadership• Revised SLP3Creating aMulti-TieredSystem ofSupport Servicesfor Literacy4www.hillforliteracy.org 2

How Do We Begin?Focus on the BIG PictureFocus on Goals and ObjectivesFocus on Structures & ProceduresFocus on Tasks/Actions5HILL Methods: Research-BasedSchool Change ProcessesWhat Works Clearinghouse: IES Practice Guides• Using Student Achievement Data to supportInstructional Decision Makinghttp://ies.ed.gov/ncee/wwc/PracticeGuide.aspx?sid=12•Assisting Students Struggling with Reading:Response to Intervention (RtI) and Multi-TierIntervention in the Primary Gradeshttp://ies.ed.gov/ncee/wwc/PracticeGuide.aspx?sid=3www.hillforliteracy.org 3

“Using Student Achievement Data toSupport Instructional Decision Making”Common Assessments:1. Statewide accountability tests.2. Commercially produced tests – including interim, benchmarkassessments, or early grade reading assessments –administered at multiple points throughout the school year.3. End-of-course tests administered across district or schools4. Interim tests developed by districts or schools that areadministered consistently and routinely to provideinformation that can be compared across classrooms orschools.IES Practice Guide, September 20095 Recommendations1. Make data part of an ongoing cycle of instructionalimprovement.2. Teach student to examine their own data and set learninggoals.3. Establish a clear vision for schoolwide data use.4. Provide supports that foster a data-driven culture within theschool.5. Develop and maintain a districtwide data system.IES Practice Guide, September 2009www.hillforliteracy.org 4

4. Provide supports that foster a datadrivenculture within the school Designate a school-based facilitator who meets withteacher teams to discuss data. Dedicate structured time for staff collaboration. Provide targeted professional development regularly.IES Practice Guide, September 2009What is the core principle?The Importance of Assessmentto Inform Instruction10www.hillforliteracy.org 5

Build an Assessment FrameworkFall Winter Spring As NeededComponentsScreening/ Benchmark/Outcomes Benchmark Outcomes / BenchmarkProgress MonitoringGroup Individual Group Individual Group Individual IndividualOral LanguagePhonological/PhonemicAwarenessNWEA-MAPfor PrimaryGradesNWEA-MAPfor PrimaryGradesNWEA-MAP forPrimary GradesPre-Reading SkillsAssessment (RGR),as needed.Phonics/Word StudyNWEA-MAPfor PrimaryGradesBeginningDecodingSurveyAssessment(RGR)NWEA-MAPfor PrimaryGradesNWEA-MAP forPrimary Grades BeginningDecoding SurveyAssessment(RGR) Pre-Reading SkillsAssessment (RGR),as needed.FluencyVocabularyNWEA-MAPfor PrimaryGradesPSF, NWF,(AIMSweb)NWEA-MAPfor PrimaryGradesPSF, NWF,ORF(AIMSweb)NWEA-MAP forPrimary GradesNWF ORF(AIMSweb)PSF, NWF, ORF(AIMSweb)ComprehensionNWEA-MAPfor PrimaryGradesNWEA-MAPfor PrimaryGradesNWEA-MAP forPrimary GradesWritingNWEA-MAPfor PrimaryGradesNWEA-MAPfor PrimaryGradesNWEA-MAP forPrimary GradesLLC p. 128= Not in all buildings= Pilot= Clarify useChange is a Process –Not an Event!Making it Real!www.hillforliteracy.org 6

Making It Real: Linking PD toLeadership Routines1. LeadershipTraining2. Prepare forImplementation3. Administration& ScoringWorkshop4. AnnualReview(Admin. &Scoring)5. DataAnalysisWorkshop6. LeadershipBenchmarkData Review7. Set District,School, ClassroomData Goals8. FallBenchmarkData Meetings9.LeadershipWalkThrough10. ProgressMonitoringMeetings11. WinterLeadershipBenchmarkReview12. WinterBenchmarkData Meetings13. SpringLeadershipBenchmarkReview14. SpringDataMeetings5 Recommendations1. Make data part of an ongoing cycle of instructionalimprovement.2. Teach student to examine their own data and set learninggoals.3. Establish a clear vision for schoolwide data use.4. Provide supports that foster a data-driven culture within theschool.5. Develop and maintain a districtwide data system.IES Practice Guide, September 2009www.hillforliteracy.org 7

1. Make data part of an ongoing cycleof instructional improvement. Collect and prepare a variety of data about studentlearning. Interpret data and develop hypotheses about how toimprove student learning. Modify instruction to test hypotheses and increasestudent learning.IES Practice Guide, September 2009Discussion of Data Sets• Individual Student Level• Classroom Level• Grade Level• District and School Level16www.hillforliteracy.org 8

Category of Reading AchievementData Sets Reading Fluency Reading ComprehensionBenchmark Strategic Intensive Benchmark Strategic IntensiveIndividualStudentClassrooms withinGrade LevelsStudent is meeting theORAL Reading Fluency(DIBELS) Benchmarkfor that grade level or isat or above the 50 th%tile on ORF NormsStudent is in thestrategic range forgrade level asidentified onDIBELSor is between the25 th and 49 th%tile (ORF norms)Student is in theintensive rangefor grade level asidentified onDIBELS or belowthe 25 th %tile(ORF Norms)The Hasbrouck and Tindal norms can be used if DIBELS norms arenot being used.75% or more ofstudents within theclassroom are meetingBenchmark Goals50-74% ofstudents withinthe classroom aremeetingBenchmark GoalsLess than 50% ofstudents aremeeting theBenchmark GoalsOn a normreferenced generalmeasure of readingcomprehension thestudent scores ator above the 60 th%tileOn a normreferencedgeneral measureof readingcomprehensionthe studentscores betweenthe 40 th and the59 th %tileOn a norm referencedgeneral measure ofreading comprehensionthe student scores belowthe 40 th %tileThe criteria may be determined by the school and will varydepending on the type of assessment and the relationship with highstakes tests or other assessments.75% or more ofstudents withinthe classroomscoring at or abovethe 60 th percentileor the 6 th stanine50-74% ofstudents withinthe classroomare scoring at orabove the 60 thpercentile or the6 th stanineLess than 50% of studentswithin the classroom arescoring at or above the60 th percentile or the 6 thstanineGrade Levels withinthe School or District[For schools with largenumbers of classroomsit may be helpful tolook at % ofclassrooms]School & DistrictLLC p. 13175% or more ofclassrooms and/orstudents in the gradelevel are meetingBenchmark Goals75% or more of allstudents K-5 aremeeting BenchmarkGoals50-74% ofclassroomsand/or studentsin the grade levelare meetingBenchmark Goals50-74% of allstudents K-5 aremeetingBenchmark GoalsLess than 50% ofclassroomsand/or studentsin the grade levelare meetingBenchmark GoalsLess than 50% ofall students K-5are meetingBenchmark Goals75% or more ofstudents withinthe grade level arescoring at or abovethe 60 th percentileor the 6 th stanine75% or more ofstudents withinthe school ordistrict are scoringat or above the 60 thpercentile or the6 th stanine50-74% ofstudents withinthe grade levelare scoring at orabove the 60 thor the 6 thstanine50-74% ofstudents withinthe school ordistrict arescoring at orabove the 60 thpercentile or the6 th stanineLess than 50% of studentswithin the grade level arescoring at or above the60 th percentile or the 6 thstanineLess than 50% of studentsin the school or districtare scoring at or above the60 th percentile or the 6 thstanineTargeted Coaching GuideBased On Percentage Of Students Meeting CBM ORF BenchmarksYear______ Grade_________FALL ORF WINTER ORF SPRING ORFEXCEEDINGBENCHMARKS80-100%BENCHMARK75 -80%MILLER 80% MILLER 80% MILLER 85JONES 75 PIRRI 76T0TAL BENCHMARK 1 2 4STRATEGIC50-74 %JONES 67BURNS 64LAND 60CARUSI 60CARR 57MCMANUS53SMITH 53LOFT52MAX 50HOOK 50RICH 71SMITH 71MAX 67NOLETTE 66HOOK 65CARUSI 62CARR 60LAND 60LOFT57HAYNES 55BURNS 53PIRRI 50MCMANUS 50TOTAL STRATEGIC 8 11 6INTENSIVEBELOW 50%NOLETTE 45HAYNES 39TRACEY 33RICH 32PIRRI 32FIFE 30LARKING 12KERBY 11PHILLIPS 0TRACEY 35FIFE 43LARKING 12KERBY 11PHILLIPS 0TOTAL INTENSIVE 9 5 7HAYNES 72CARUSI 71LOFT70RICH 70SMITH 63NOLETTE 60BURN 60LAND 60JONES 59MCMANUS 50CARR 50HOOK 45MAX 42TRACEY 35FIFE 29LARKING 12KERBY 11PHILLIPS 0Study Trends – DevelopCoaching Strategieswww.hillforliteracy.org 9

Grouping Studentsfor “WIN” InstructionFALL DIBELS - GRADE 1 - INSTRUCTIONAL GROUPINGTEACHER:Score Letter Naming Phoneme Segmentation Nonsense Word Instructional Support / Providers37+ Jhonelle; Kimberly Juan Frank37+ Ruben; Philip37+37+37+37363534 Anensky Jhonelle; Xavier; Philip33323130 Mathew; Jorge29 Xavier28 Rachel; Frank27 Aliya RubenStem & Leaf Charts26 Thalia25 Anthony; Eric; Tanisha24 Rachel23 Juan Jhonelle; Philip22 Frank21 Rachel20 Thalia Anensky; Kimberly191817 Ruben1615 Kenny Eric14 Tasha Anthony; Tasha; Kimberly13 Star Kenny12 Anthony; Kenny11 Aliya Anensky10 Tanisha9 MathewJeffrey8 Xavier7 Tasha6 Juan5 Mathew; Jeffrey4 Aliya3 Tanisha210 Malik; Alexi Star; Malik; Alexi; JorgeEric; Thalia; Star; Jeffrey;Malik; Alexi; JorgeLLC p. 148Low Risk CT = Classroom Teacher TI = Title I TeacherSome Risk SPED=Special Ed Teacher P = ParaprofessionalAt RiskRS = Reading Specialistwww.hillforliteracy.org 10

Labour Movement and DemocratisationA number of previous studies have noted how Indonesian labour was unableto assert its power during the early democratisation period the country wasexperiencing (See Hadiz 1998, Aspinal 1999, Törnquist 2004). 2 Their analysesbegin with a comparative perspective on the documented experiences oflabour’s actions and experiences in other post-authoritarian societies. Indeed,in such comparisons, Indonesian labour’s contribution to democratisation inthe early years of Reformasi was minimal and limited. Under the context ofthe Reformasi, Indonesian labour failed to advance its interests during thedemocratisation process.Looking beyond the particularities, the Indonesian situation ishardly unique. Similar situations wherein labour was unable to make itscollective voice heard during the democratisation process also happenedin some African countries. In Nigeria, unions failed to take part in thedemocractisation process (See Beckmann and Sachikonye 2010). Even whenthe labour movement played a significant role, such as in Niger and Ghana,the intensity of their involvement in the political democratisation of thecountry dwindled and became directionless, leading many labour unions toquestion the advantages of democracy (see Kraus 2007). Scholars have alsonoted how the once celebrated Solidarność which played a central role inthe democratisation of Poland slowly lost its ground in the later stages ofdemocratisation, especially after the neoliberal’s “shock therapy” (See Ost,2005). These studies show how the timing of democratisation provides amplespace for labour to maneuver, but it does not guarantee labour’s continuingparticipation in the later stage of the democratisation process. Labour isoften found defeated or viewed as obstacle in the later stage.The limited participation of Indonesian labour in the early stage ofdemocratisation is due to a number of factors. The most obvious one isthe fragmentation and factionalism of the movement: as many as 80 unionfederations were established during the first five years of the Reformasi.Unable to form a united front and coordinate their actions, unions failed toconsolidate their strength and utilise the political space provided with theintroduction of democracy. Fortunately, recently there has been a strong moveamong union leaders to start building common ground, given the sharedchallenges of their struggle for socio-political recognition. In that context ofunions’ consolidation efforts, Surya Tjandra’s article, “The Indonesian trade2 Worker activism after reformasi 1998

Grouping Students by SkillsORF Accuracy Comprehension/VocabularyStudentsReader Profile On and above level ReadersReader Profile 1:Automatic Word CallersReader Profile 2:Struggling Word CallersReader Profile 3:Word StumblersReader Profile 4:Slow ComprehendersReader Profile 5:Slow Word Callers Reader Profile 6:Severely Disabled ReadersDeveloped by: A. Janoch & A. Allwarden, 2012. ConVal School Districtwww.hillforliteracy.org 12

Grouping Students byReader ProfilesStudent ORF ORF Accuracy DazeNWEAWord IDNWEALiteracyTextsNWEAInfo. TextsEmily 173 98% 23 HI HI HIKate 142 100% 26 HI HI HICathy 141 99% 27 HI AV AVJack 136 100% 26 HI HI HIEric 132 96% 22 AV AV HIKatherine 129 99% 22 HI HI HIDavid 126 98% 21 HI HI HIMartha 115 99% 21 HI HI AVChristine 100 98% 13 HI AV AVAlexis 97 98% 12 HI AV HIMatthew 95 99% 15 HI HI HIDaniel 90 99% 19 AV LO LOHenry 89 86% 15 AV HI AVMary 18 75% 3 LO LO LOSally 16 67% 7 LO LO LOORF Accuracy Comprehension/VocabularyStudentsReader Profile Emily, Kate, Cathy, Jack, Eric,Katherine, David, MarthaHenryOn and above level ReadersReader Profile 1:Automatic Word CallersReader Profile 2:Struggling Word CallersReader Profile 3:Word StumblersChristine, Alexis, MatthewReader Profile 4:Slow ComprehendersDanielReader Profile 5:Slow Word Callers Mary, SallyReader Profile 6:Severely Disabled Readerswww.hillforliteracy.org 13

Reader Profile/InstructionalFocus MatrixReader1Reader2Reader3Reader4Reader5Reader6BeginningDecodingNeedsWord Work Fluency Vocabulary ComprehensionBasic SightWordsMultisyllableWordStrategiesRate Prosody AcademicVocabularyBreadth ofVocabularyVocabularyStrategiesSelf‐Monitoring:Fix‐UpStrategiesSelf‐MonitoringMeaningRetell/SummarizeTextStructures● ● ● ●● ● ● ● ● ●●Henry●●Henry●ChristineAlexisMatthew●? ● ● ●●MarySally●●MarySallyDeveloped by: A. Janoch & A. Allwarden, 2012. ConVal School District• Assessing StudentsEssential to SystematizeProcedures & Protocols• Grouping Studentswww.hillforliteracy.org 14

Strategic Literacy Implementation (I) & Sustainability (S) PlanDistrict/School Name: ConVal (SAU #1)Grades: K-5Date Developed: 6.21.11 SY: 2011-2012Dates Reviewed: 10.4.11; 11.9.11; 1.16.12; 2.21.12; 3.16.12; 4.17.12; 5.17.12; 6.14.12PRIORITY GOAL 1: Develop and implement an Assessment Framework that aligns with Response to Intervention (RtI) multi-tiers of instruction and includes measures in all of the key components of literacy.ACTION1.1 (I) Download and organize material forAIMSweb administration and scoring foreach school/classroom.RESPONSIBLEPERSON(S)Test assessment teamDesignated building level staff.START COMPLETION DATERESOURCESDATENEEDED7.15.11 8.29.11 AIMSweb site.Copy machine, paperMEASURES OFSUCCESSAdequate materials to begin FallBenchmarkingStudent lists compiled and enteredAPPRAISALCompleted and @ school has a benchmarkbinder. Buildings will download their PMmaterials. Completed.1.2 (I) Database (migrate students within and across Carol Arnold 9.15.11 10.1.11 AIMSweb site All students have been migrated to theAIMSweb site.schools, enter new students and teachers,etc.). Determine database access forteachers.Students are migrated. Teachers can accessdata. Completed. Consider pairing Carol witha partner for this.1.3 (I) Meet with assessment teams to finalizeadministration schedule and materials.Saunders, Allwarden, Janoch 8.25.11 8.25.11 Materials ListCalendarAssessment Schedule developed anddisseminatedCompleted. Trained teachers will join theassessment team for winter benchmark.Some schools did their own assessments inSpring.1.4 (I) Train teachers and support staff inadministration and scoring of AIMSweb.Test assessment teamDesignated building level staff.All staff for Progress Monitoring8:25.11 9.30.11 AIMSweb Presenter, <strong>Handout</strong>s Adequate materials to begin ProgressMonitoring.Student lists compiled and enteredCompleted 8.26.11 during District-wide PDDay. Web-based training beginning Oct onPM. Schedule new staff next year. Establishstructures for inter-rater reliability checks.1.5 (S) Develop a calendar for administration andscoring of benchmark and progressmonitoring assessments.1.6 (I) Establish a building level person to checkmonthly that benchmark and progressmonitoring scores are being entered in thesystem.1.7 (I) Offer pre-data meeting sessions to helpteachers review and analyze class data(AIMSweb)Saunders, Allwarden, Janoch 8.30.11 9.15.12 AIMSweb manual; assessment staff Calendar Placed in Dropbox and Google Calendar andalerted principals 9/1/11Assistant Superintendent to assign 9.15.11 6.30.12 AIMSweb database Schedule developed and followed. Need to do ASAP. Not done.person(s). Principals will followup.Coaches 9.15.11 10.15.11 AIMSweb databaseMeeting Notes completed and Didn’t happen for Fall due to just startingComprehensive SpreadsheetdisseminatedAIMSweb. Not sure if we need this actionstep. Conducted grouping chart professionaldevelopment. Should continue – principalsneed to hold teachers accountable.1.8(I)Conduct Data Meetings on a regular basisthroughout the year with grade-level teams(to be co-facilitated by Principal andTeachers). Use the data meeting protocols.Principals, Teachers, supported bycoaches10.3.11 10.14.11 AIMSweb reports, HILL protocols,grouping charts, instructional strategiesCompleted grouping chartsData Meetings to be set and conductedwithin two weeks (10.4.11). Check withprincipals.1.9 (S) Continue consistent use of the dataspreadsheet that includes all Districtassessments to set goals during datameetings.Allwarden, Literacy CoachSaunders, Jones9.7.11 6.29.12 SpreadsheetPerformance PlusAppropriate differentiated studentgroupingsCheck Performance Pathways to see if it canbe done like the spreadsheet.Not being used across the district – discuss.1.10 (I) Complete the Assessment Rubric, coordinatework between RtI and District LiteracyLeadership Team to establish anddisseminate a District AssessmentFramework K-4Saunders, District LLTCommittee, RtI Committee,Literacy Coaches, Jones9.7.11 6.29.12 Continuum of Service Delivery Framework,Assessment reviews, Draft assessment document.District Assessment FrameworkOngoing – discuss with PrincipalsAnytown School Grade 2 DataGRADE 2 Oral Reading Fluency Accuracy Comprehension PhonicsSPRINGSpringADSStudentNameTeacherName Fall WinterSpring(AR)List.CompPassageComp#correctCorrect Words Per Minute Stanine Max=33Bryan Jones 39 80 83 99%34 27Caitlynn Jones - 37 56 90%5- 13*David Jones 12 30 57 93%52 11*Joshua Jones 13 31 38 88%32 13*Kelli Jones 70 67 88 97%65 25Kevin Jones 54 72 87 94%4- 21Madison Jones 18 37 56 92% 4 2 17Melissa Jones 110 100%66 30Siena Jones 129 96%43 26Tabitha Jones 27 44 54 87%53 7*Trevor Jones 160 98%98 33Zach G. Jones 70 71 102 93%73 14*Zach J Jones 25 33 53 86%82 6*Alex Jones 137 93%75 27Brainna Jones 69 73 96 100%44 28Chris Jones 73 108 121 98%55 28Spring fluency score(correct words/minute):Benchmark= 111+Strategic= 84 - 110Intensive=

State Outcome AssessmentGrade-level/Classroom Student Data SummaryStudentName Grade TeacherBOYORFMOYORFEOYORFBOYCompEOYCompUnitTestPlacement TestStateOutcome Other Other31Strategic Literacy Implementation (I) & Sustainability (S) PlanDistrict/School Name: Sample Rural DistrictGrades: K-5Date Developed: 6.21.11 SY: 2011-2012Dates Reviewed: 10.4.11; 11.9.11; 1.16.12; 2.21.12; 3.16.12; 4.17.12; 5.17.12; 6.14.12PRIORITY GOAL 1: Develop and implement an Assessment Framework that aligns with Response to Intervention (RtI) multi-tiers of instruction and includes measures in all of the key components of literacy.ACTION1.1 (I) Download and organize material forAIMSweb administration and scoring foreach school/classroom.RESPONSIBLEPERSON(S)Test assessment teamDesignated building level staff.START COMPLETION DATERESOURCESDATENEEDED7.15.11 8.29.11 AIMSweb site.Copy machine, paperMEASURES OFSUCCESSAdequate materials to begin FallBenchmarkingStudent lists compiled and enteredAPPRAISALCompleted and @ school has a benchmarkbinder. Buildings will download their PMmaterials. Completed.1.2 (I) Database (migrate students within and across Admin 9.15.11 10.1.11 AIMSweb site All students have been migrated to theAIMSweb site.schools, enter new students and teachers,etc.). Determine database access forteachers.Students are migrated. Teachers can accessdata. Completed. Consider pairing Carol witha partner for this.1.3 (I) Meet with assessment teams to finalizeadministration schedule and materials.Asst Superintendent, District LiteracyCoaches8.25.11 8.25.11 Materials ListCalendarAssessment Schedule developed anddisseminatedCompleted. Trained teachers will join theassessment team for winter benchmark.Some schools did their own assessments inSpring.1.4 (I) Train teachers and support staff inadministration and scoring of AIMSweb.Test assessment teamDesignated building level staff.All staff for Progress Monitoring8:25.11 9.30.11 AIMSweb Presenter, <strong>Handout</strong>s Adequate materials to begin ProgressMonitoring.Student lists compiled and enteredCompleted 8.26.11 during District-wide PDDay. Web-based training beginning Oct onPM. Schedule new staff next year. Establishstructures for inter-rater reliability checks.1.5 (S) Develop a calendar for administration andscoring of benchmark and progressmonitoring assessments.Asst Superintendent, DistrictLiteracy Coaches8.30.11 9.15.12 AIMSweb manual; assessment staff Calendar Placed in Dropbox and Google Calendar andalerted principals 9/1/111.6 (I) Establish a building level person to checkmonthly that benchmark and progressmonitoring scores are being entered in thesystem.1.7 (I) Offer pre-data meeting sessions to helpteachers review and analyze class data(AIMSweb)Assistant Superintendent to assign 9.15.11 6.30.12 AIMSweb database Schedule developed and followed. Need to do ASAP. Not done.person(s). Principals will followup.Coaches 9.15.11 10.15.11 AIMSweb databaseMeeting Notes completed and Didn’t happen for Fall due to just startingAIMSweb. Not sure if we need this actionComprehensive Spreadsheetdisseminatedstep. Conducted grouping chart professionaldevelopment. Should continue – principalsneed to hold teachers accountable.1.8(I)Conduct Data Meetings on a regular basisthroughout the year with grade-level teams(to be co-facilitated by Principal andTeachers). Use the data meeting protocols.Principals, Teachers, supported bycoaches10.3.11 10.14.11 AIMSweb reports, HILL protocols,grouping charts, instructional strategiesCompleted grouping chartsData Meetings to be set and conductedwithin two weeks (10.4.11). Check withprincipals.1.9 (S) Continue consistent use of the dataspreadsheet that includes all Districtassessments to set goals during datameetings.Asst Superintendent, DistrictLiteracy Coaches, HILL9.7.11 6.29.12 SpreadsheetPerformance PlusAppropriate differentiated studentgroupingsCheck Performance Pathways to see if it canbe done like the spreadsheet.Not being used across the district – discuss.1.10 (I) Complete the Assessment Rubric, coordinatework between RtI and District LiteracyLeadership Team to establish anddisseminate a District AssessmentFramework K-4Asst. Superintendent, DistrictLLT Committee, RtI Committee,Literacy Coaches, HILL9.7.11 6.29.12 Continuum of Service Delivery Framework,Assessment reviews, Draft assessment document.District Assessment FrameworkOngoing – discuss with Principalswww.hillforliteracy.org 16

No Evidence ofImplementationAssessment frameworkThe district does not havea literacy assessmentframework.Minimal Evidence ofImplementationThe district has anincomplete assessmentframework (many gapsand/or redundancies).DevelopingImplementationThe district has a literacyassessment frameworkacross schools and grades,but the data are notroutinely examined orused by many teachers.Assessment RubricProficientImplementationThe district has a literacyassessment frameworkacross schools and gradesand data are routinelyused by most teachers toguide instruction andintervention.ExemplaryImplementationThe district has a literacyassessment frameworkacross schools and gradesand data are routinelyused by all teachers toguide instruction andintervention. P‐4 5‐8 9‐12District assessmentsThe district does notrequire that schoolsadminister consistentformative and summativeassessments to allstudents in a timelymanner in addition to thestate outcome ELAassessment. P‐4 5‐8 9‐12The district requires thatschools administer someformative and summativeassessments to students in atimely manner in addition tothe state outcome ELAassessment. P‐4 5‐8 9‐12The district requires thatschools administerformative and summativeassessments (using anincomplete assessmentframework) to somestudents in a timelymanner in addition to thestate outcome ELAassessment. P‐4 5‐8 9‐12The district requires thatschools administerformative and summativeassessments (using acomplete assessmentframework) to moststudents in a timelymanner in addition to thestate outcome ELAassessment. P‐4 5‐8 9‐12District ensures thatschools administerconsistent formative andsummative assessments(using a fully developedAssessment Framework)to all students in a timelymanner in addition to thestate outcome ELAassessment. P‐4 5‐8 9‐12Data storageThe district does notprovide a data storagesystem to schools. P‐4 5‐8 9‐12The district provides a limiteddata storage system toschools and only a few userscan employ. P‐4 5‐8 9‐12The district provides acomprehensive datastorage system but only afew users can employ. P‐4 5‐8 9‐12The district provides acomprehensive datastorage system and someusers can employ it. P‐4 5‐8 9‐12The district provides acomprehensive datastorage system and allusers can employ it. P‐4 5‐8 9‐12 P‐4 5‐8 9‐12 P‐4 5‐8 9‐12 P‐4 5‐8 9‐12 P‐4 5‐8 9‐12Reflect…• School groups complete the rubric together.(5 min)• Identify 3 areas that you think you shouldwork on right away.• Explain your rationale for your selections.• How would you use this tool?www.hillforliteracy.org 17

Strategic Literacy Implementation (I) & Sustainability (S) PlanDistrict/School Name: Sample Rural DistrictGrades: K-5Date Developed: 6.21.11 SY: 2011-2012Dates Reviewed: 10.4.11; 11.9.11; 1.16.12; 2.21.12; 3.16.12; 4.17.12; 5.17.12; 6.14.12PRIORITY GOAL 1: Develop and implement an Assessment Framework that aligns with Response to Intervention (RtI) multi-tiers of instruction and includes measures in all of the key components of literacy.ACTION1.1 (I) Download and organize material forAIMSweb administration and scoring foreach school/classroom.RESPONSIBLEPERSON(S)Test assessment teamDesignated building level staff.START COMPLETION DATERESOURCESDATENEEDED7.15.11 8.29.11 AIMSweb site.Copy machine, paperMEASURES OFSUCCESSAdequate materials to begin FallBenchmarkingStudent lists compiled and enteredAPPRAISALCompleted and @ school has a benchmarkbinder. Buildings will download their PMmaterials. Completed.1.2 (I) Database (migrate students within and across Admin 9.15.11 10.1.11 AIMSweb site All students have been migrated to theAIMSweb site.schools, enter new students and teachers,etc.). Determine database access forteachers.Students are migrated. Teachers can accessdata. Completed. Consider pairing Carol witha partner for this.1.3 (I) Meet with assessment teams to finalizeadministration schedule and materials.Asst Superintendent, District LiteracyCoaches8.25.11 8.25.11 Materials ListCalendarAssessment Schedule developed anddisseminatedCompleted. Trained teachers will join theassessment team for winter benchmark.Some schools did their own assessments inSpring.1.4 (I) Train teachers and support staff inadministration and scoring of AIMSweb.Test assessment teamDesignated building level staff.All staff for Progress Monitoring8:25.11 9.30.11 AIMSweb Presenter, <strong>Handout</strong>s Adequate materials to begin ProgressMonitoring.Student lists compiled and enteredCompleted 8.26.11 during District-wide PDDay. Web-based training beginning Oct onPM. Schedule new staff next year. Establishstructures for inter-rater reliability checks.1.5 (S) Develop a calendar for administration andscoring of benchmark and progressmonitoring assessments.Asst Superintendent, DistrictLiteracy Coaches8.30.11 9.15.12 AIMSweb manual; assessment staff Calendar Placed in Dropbox and Google Calendar andalerted principals 9/1/111.6 (I) Establish a building level person to checkmonthly that benchmark and progressmonitoring scores are being entered in thesystem.1.7 (I) Offer pre-data meeting sessions to helpteachers review and analyze class data(AIMSweb)Assistant Superintendent to assign 9.15.11 6.30.12 AIMSweb database Schedule developed and followed. Need to do ASAP. Not done.person(s). Principals will followup.Coaches 9.15.11 10.15.11 AIMSweb databaseMeeting Notes completed and Didn’t happen for Fall due to just startingAIMSweb. Not sure if we need this actionComprehensive Spreadsheetdisseminatedstep. Conducted grouping chart professionaldevelopment. Should continue – principalsneed to hold teachers accountable.1.8(I)Conduct Data Meetings on a regular basisthroughout the year with grade-level teams(to be co-facilitated by Principal andTeachers). Use the data meeting protocols.Principals, Teachers, supported bycoaches10.3.11 10.14.11 AIMSweb reports, HILL protocols,grouping charts, instructional strategiesCompleted grouping chartsData Meetings to be set and conductedwithin two weeks (10.4.11). Check withprincipals.1.9 (S) Continue consistent use of the dataspreadsheet that includes all Districtassessments to set goals during datameetings.Asst Superintendent, DistrictLiteracy Coaches, HILL9.7.11 6.29.12 SpreadsheetPerformance PlusAppropriate differentiated studentgroupingsCheck Performance Pathways to see if it canbe done like the spreadsheet.Not being used across the district – discuss.1.10 (I) Complete the Assessment Rubric, coordinatework between RtI and District LiteracyLeadership Team to establish anddisseminate a District AssessmentFramework K-4Asst. Superintendent, DistrictLLT Committee, RtI Committee,Literacy Coaches, Jones9.7.11 6.29.12 Continuum of Service Delivery Framework,Assessment reviews, Draft assessment document.District Assessment FrameworkOngoing – discuss with PrincipalsAnytown School Assessment CalendarPut Assessment Scheduleon the School Calendar!= Benchmark= ProgressMonitoringwww.hillforliteracy.org 18

Strategic Literacy Implementation (I) & Sustainability (S) PlanDistrict/School Name: Sample Rural DistrictGrades: K-5Date Developed: 6.21.11 SY: 2011-2012Dates Reviewed: 10.4.11; 11.9.11; 1.16.12; 2.21.12; 3.16.12; 4.17.12; 5.17.12; 6.14.12PRIORITY GOAL 1: Develop and implement an Assessment Framework that aligns with Response to Intervention (RtI) multi-tiers of instruction and includes measures in all of the key components of literacy.ACTION1.1 (I) Download and organize material forAIMSweb administration and scoring foreach school/classroom.RESPONSIBLEPERSON(S)Test assessment teamDesignated building level staff.START COMPLETION DATERESOURCESDATENEEDED7.15.11 8.29.11 AIMSweb site.Copy machine, paperMEASURES OFSUCCESSAdequate materials to begin FallBenchmarkingStudent lists compiled and enteredAPPRAISALCompleted and @ school has a benchmarkbinder. Buildings will download their PMmaterials. Completed.1.2 (I) Database (migrate students within and across Admin 9.15.11 10.1.11 AIMSweb site All students have been migrated to theAIMSweb site.schools, enter new students and teachers,etc.). Determine database access forteachers.Students are migrated. Teachers can accessdata. Completed. Consider pairing Carol witha partner for this.1.3 (I) Meet with assessment teams to finalizeadministration schedule and materials.Asst Superintendent, District LiteracyCoaches8.25.11 8.25.11 Materials ListCalendarAssessment Schedule developed anddisseminatedCompleted. Trained teachers will join theassessment team for winter benchmark.Some schools did their own assessments inSpring.1.4 (I) Train teachers and support staff inadministration and scoring of AIMSweb.Test assessment teamDesignated building level staff.All staff for Progress Monitoring8:25.11 9.30.11 AIMSweb Presenter, <strong>Handout</strong>s Adequate materials to begin ProgressMonitoring.Student lists compiled and enteredCompleted 8.26.11 during District-wide PDDay. Web-based training beginning Oct onPM. Schedule new staff next year. Establishstructures for inter-rater reliability checks.1.5 (S) Develop a calendar for administration andscoring of benchmark and progressmonitoring assessments.Asst Superintendent, DistrictLiteracy Coaches8.30.11 9.15.12 AIMSweb manual; assessment staff Calendar Placed in Dropbox and Google Calendar andalerted principals 9/1/111.6 (I) Establish a building level person to checkmonthly that benchmark and progressmonitoring scores are being entered in thesystem.1.7 (I) Offer pre-data meeting sessions to helpteachers review and analyze class data(AIMSweb)Assistant Superintendent to assign 9.15.11 6.30.12 AIMSweb database Schedule developed and followed. Need to do ASAP. Not done.person(s). Principals will followup.Coaches 9.15.11 10.15.11 AIMSweb databaseMeeting Notes completed and Didn’t happen for Fall due to just startingAIMSweb. Not sure if we need this actionComprehensive Spreadsheetdisseminatedstep. Conducted grouping chart professionaldevelopment. Should continue – principalsneed to hold teachers accountable.1.8(I)Conduct Data Meetings on a regular basisthroughout the year with grade-level teams(to be co-facilitated by Principal andTeachers). Use the data meeting protocols.Principals, Teachers, supported bycoaches10.3.11 10.14.11 AIMSweb reports, HILL protocols,grouping charts, instructional strategiesCompleted grouping chartsData Meetings to be set and conductedwithin two weeks (10.4.11). Check withprincipals.1.9 (S) Continue consistent use of the dataspreadsheet that includes all Districtassessments to set goals during datameetings.Asst Superintendent, DistrictLiteracy Coaches, HILL9.7.11 6.29.12 SpreadsheetPerformance PlusAppropriate differentiated studentgroupingsCheck Performance Pathways to see if it canbe done like the spreadsheet.Not being used across the district – discuss.1.10 (I) Complete the Assessment Rubric, coordinatework between RtI and District LiteracyLeadership Team to establish anddisseminate a District AssessmentFramework K-4Asst. Superintendent, DistrictLLT Committee, RtI Committee,Literacy Coaches, HILL9.7.11 6.29.12 Continuum of Service Delivery Framework,Assessment reviews, Draft assessment document.District Assessment FrameworkOngoing – discuss with PrincipalsData Meeting ProtocolsFall Data Meeting ProtocolLiteracy Initiative – Level 1Stage 1: Organizing for the MeetingStep Action MaterialsNeeded1 Choose MeetingLeader1.1 Assign Facilitator 1.1.1 Meeting AgendaTemplatePurpose/RationaleKeep group on track and disseminatemeeting notesResponsiblePersonCoach orPrincipal2 Take MeetingNotes2.1 Assign Scribe 2.1.1 Template for MeetingNotesProvide documentation of decisions,action steps and timelineCoach orPrincipal3 Gather DataResults & Bringto Meeting, asappropriate3.1 Early Literacy Comprehension & Fluency Benchmarks, Core Program Baseline Test, if given3.2 Fluency Database Reportsand/or Core ProgramAssessments3.2.1 Summary ReportsFrom Databaseand/or Core ProgramDetermine what percentage of students ineach grade level are At Risk, Some Risk,Low Risk or, Established, Emerging, Deficit3.3 Progress MonitoringCaseload Organizers3.4 Bring Test Booklets ofIntervention Students3.2.2 Class Lists, scores ofall studentsorganized by class3.2.3 Ind. History Rpts forselect students3.3.1 Data system reports,orDetermine what percentage of students ineach class are At Risk, Some Risk, Low Riskor, Established, Emerging, DeficitDetermine which individual students are ina deficit situationAssist teachers to organize and trackstudents needing PM3.3.2 Caseload Organizer Assist teachers to organize and trackstudents needing PM3.4.1 Student Booklets Allows for deeper analysis of individualstudentsCoach &TeacherCoach &TeacherCoach &TeacherCoachCoachTeacher &Coach3.5 Comprehension Scores,Surveys, Diagnostic Scores,etc.3.5.1 Comprehensive DataSpreadsheet3.6 Instructional Resources 3.6.1 Curriculum/ ProgramMaterials, etc.Provides data for a classroom or gradelevel at a glance.Provides quick review of instructionalstrategiesCoach,Principal &TeacherCoach &Teacherwww.hillforliteracy.org 19

Fall DataMeetingProtocolsRelease ofResponsibilityLevel 1Stage 1:Organizing for MeetingStage 2:Pre‐Meeting AnalysisStage 3:Conducting the MeetingLevel 2Stage 1:Organizing for MeetingStage 2:Pre‐Meeting AnalysisStage 3:Conducting the MeetingLevel 3Stage 1:Organizing for MeetingStage 2:Pre‐Meeting AnalysisStage 3:Conducting the MeetingLevel 4Stage 1:Organizing for MeetingStage 2:Pre‐Meeting AnalysisStage 3:Conducting the MeetingAction Steps Responsible Person Support Person(s)1 – 2 Coach Principal3 Teacher4 –5 Coach Principal6 ‐ 11 Coach PrincipalTeacher1 – 2 Principal/Teacher Coach3 Teacher/Principal Coach4 – 7 Principal/Teacher Coach8 Teacher/Coach Principal9 ‐ 12 Teacher Principal/Coach1 ‐ 3 Teacher Principal/Coach*4 ‐ 7 Teacher Principal/Coach*8 –11 Teacher Principal/Coach*1 ‐ 3 Teacher Principal*4 ‐ 7 Teacher Principal/Coach*8 –11 Teacher Teacher TeamUsing a Protocol to GroupStudentsTeacher:Grade:Time:Group:Tiered Classroom and Grade Level Grouping ChartAssessmentCriteria/RangeInstructional Focus/ Priority SkillOther:(Program/Materials;Progress MonitoringInformation)Time:Group:Time:Group:Time:Group:Review Date and Progress Review Notes:LLC p. 142www.hillforliteracy.org 20

3 Tier Classroom and Grade Level Grouping Chart ExamplesTeacher: Ms. SmithGrade: 2Time:Group: Intensive(Names)Time:Group: Strategic(Names)Time:Group: Benchmark(Names)Time:Group: Benchmark(Names)AssessmentCriteria/Rangeof scores for…DIBELSDecoding SurveysNWEAHigh-FrequencyDIBELSDecoding SurveysNWEAHigh-FrequencyDIBELSDecoding SurveysNWEAHigh-FrequencyDIBELSDecoding SurveysNWEAHigh-FrequencyInstructional Focus/Priority SkillPA = Segmenting and blending phonemesP = CVCC, CCVC (short vowels, blends, digraphs)F= Automaticity of words with short vowels, blends, digraphsV = High-frequency wordsC = RetellingPA = Segmenting and blending phonemesP = VCE (silent-e pattern)F = Accuracy and rate (repeated reads)V = High-frequency wordsC = RetellingPA = Manipulating phonemesP = Review whole group focus (vowel digraphs)F = Prosody (phrasing, pausing at punctuation)V = Literacy Text (Tier 2 words), Info text (contentspecific words)C = Literacy Text (Story Elements), Info. Text(Obtaining info. from text features)PA = Manipulating phonemesP = Review whole group instruction (vowel diagraphs)F = Prosody (intonation)V = Literacy Text (Tier 2 words), Info. Text (content specificwords)C = Summarizing (with graphic organizer)Program/Materials;Progress MonitoringInformationProgress Monitor – DIBELSORF, Decoding SurveyMaterials – phonogram cards,letter tiles, decodable textMy Sidewalks, SPIRE,BOOST, LIPS, FCRR <strong>Center</strong>sProgress Monitor – DIBELSORF, Decoding SurveyMaterials – phonogram cards,letter tiles, decodable textsRS Intervention Kit, Lexia,PALS, BLITZ, FCRR <strong>Center</strong>sProgress Monitor –Comprehension Teacher –Mode and Vocabulary CBMsMaterials – sound/spellingcards, cards for word sorts,leveled text Reading Street,Lexia, FCRR <strong>Center</strong>sProgress Monitor –Comprehension andVocabulary CBMsMaterials – sound/spellingcards, cards for word sorts,leveled text Reading Street,Lexia, FCRR <strong>Center</strong>sReview Date and Progress Review Notes:Instruction and ResourceAllocationFall DIBELS NextReadingStreetBaseline TestFirst NameTeacherLNF PSF NWF TTCLS WWR Instructional Focus Program/InterventionistMaddison Bodendorf, M 8 1 0Jaden Bodendorf, M 16 0 0Aaron Bodendorf, M 45 58 18Trenten Bodendorf, M 47 35 0Kaitlyn Bodendorf, M 62 61 11Antonio Bodendorf, M 38 13 0Liberty Bodendorf, M 49 35 0Joshua Bodendorf, M 51 46 15Giovanni Bodendorf, M 18 0 0Lily Bodendorf, M 43 25 9Arianna Bodendorf, M 59 58 17Matthew Bodendorf, M 43 29 1Savana Bodendorf, M 67 68 0Gracelyn Bodendorf, M 44 33 1Nicholas Bodendorf, M 50 47 10Nunzio Bodendorf, M 38 29 0Skyla Bodendorf, M 44 18 0Kelly Bodendorf, M 39 11 0Maeve Bodendorf, M 21 36 1Dylan Bodendorf, M 44 31 0Meredith Bodendorf, M 49 38 11Zackery Bodendorf, M 64 132 45Wilfredo Bodendorf, M 37 10 0Taylor Bodendorf, M 50 40 0Jovera Bodendorf, M 49 18 2Samantha Bodendorf, M 77 61 2Baylor Bodendorf, M 43 20 0www.hillforliteracy.org 21

What Is A Data Wall?•A visual means to look at past performance.•A trend identifier.•A way to plan future assistance tracks forstudents. (multi-tiered)•A method to share with others (collaborate) thestrategies that are working for your students.•An organizational tool.Why Are Data Walls Important?They allow us to ....•identify specific areas of student need.•collaborate with others to share and learnabout new strategies.•share our success with team members and/orthe school’s faculty.•foster mutual responsibility for studentsuccess.www.hillforliteracy.org 22

Why Should We UseData Walls?They help us to ...•understand the individual student better .•stand back and look at the BIG PICTURE.•Meet the individual needs of each student(multi-tiered)Putting NamesSchool Data Walls& Faces on the Datawww.hillforliteracy.org 23

Constructing A Good Data WallMy Students’ Prior DataImproving Student Achievement6 th Grade ReadingPrior Data ObservationIntervention PlanMy Students’Intervention DataStudents onaveragescored belowthe Stateaverage in allareas of theReadingassessment.Students willreceive 15 extraminutes ofintensiveinstructionevery other dayin 2 of theirareas ofweakness.Each of theteachers in thisPLC will target 2areas ofinstruction.Students will go tothat area each day.Commonassessments willbe given weekly.SMART GoalStudents willincrease theirpoints earnedby 25% onthe 2010FCATReadingAssessmentMrs. AdlarChris D., Bobby L.,Mike T.Mrs. MillerCyndy M., Brady K.,Mike L., Bea C.,Mr. CotterWilliam M., Betsy S.,Virginia Y.Mr. SmithConnie D., TommyL., Mike T.Mrs. PetersonCally M., Btewart K.,Monica L., Conrad B.,ConclusionStudents aremakingprogress ineach area asreported by thecommonassessmentsgiven eachweek.Presented by: Art TweedieThe Office of Research, Evaluation & AccountabilityData WallsConversations about Studentswww.hillforliteracy.org 24

Web Sites to Note• www.dibels.uoregon.edu• www.hillforliteracy.org• www.FCRR.org• www.centeroninstruction.org• www.texasreading.org• www.sopriswest.com“When you have a great anddifficult task, something perhapsalmost impossible, if you work alittle at a time, every day a little,suddenly the work will finish itself.”Isak Dinesenwww.hillforliteracy.org 25