Shaping the Future - Jefferson County Public Schools

Applying an Iterative Evaluation- 

based Professional Development 

Model to Elementary Literacy 

Instruction 

Dena Dossett, Ph.D 

Beverly Winsch, Ph.D 

Jefferson County Public Schools, KY 

Jefferson County 

Public Schools 

Shaping the Future

Jefferson County Public 

Schools 

• 28 th largest school district in U.S. 

• 98,000 students (1/7 of all students in KY) 

• 153 Schools 

• Preschool…………… 

3- and 4-year4 

year-olds 

• K-5 5 (Elementary)………… 

………….. 87 schools 

• 6-88 (Middle)……………… 

……………….. 23 schools 

• 9-12 (High)………………… 

…………………. 20 schools 

• Special Education………… 

…………. 3 schools 

• Alternative/Other………… 

…………. 20 schools 




Jefferson County Public 

Schools 

Racial Composition 

34% African-American 

59% White 

7% Other 

Non-English Language 

Background students 

 

4,128 (4.2%) students 

823% growth since 1990 

 

40% of State NELB 

Exceptional Child 

Education 

13,595 students ages 3-21 

Home/Hospital students 

Homeless students 

4,600 (5%) students 

288% growth since 1993 

Single-parent households 

54% of total JCPS population 

82% of the JCPS African-American 

population 

Free or reduced-price lunch 

61% of Elementary students 

4% increase since 2002-03 




Every 1 Reads 

• Every 1 Reads is an education initiative that is 

designed to have every JCPS student reading at 

grade level within four years. 

• This goal will position the Louisville Metro 

area as a national pacesetter in education. 

• Currently, 18,000 students are 

not reading on grade level. 




JCPS Elementary Literacy 

Initiative 

Assessment 

‣ Diagnostic and achievement tests at each grade level 

Curriculum 

‣ Established, research-based, standardized instruction 

for reading and writing 

‣ Prescriptive 

‣ Core Content Guides 

‣ 38 Literacy coaches 

Intervention 

‣ Extended School Services 

‣ Tier I, II, III (additional time, supplemental materials, accommodations) 





Initiative 

Structure 

‣ Additional Time 

‣ Group, Regroup 

Professional Development 

‣ Narrow and focused 

‣ Long-term 

‣ Job-embedded 

‣ Data-driven 

‣ Research-based 

‣ All teachers trained in reading strategies 





Initiative 

Assessment 

Curriculum/ 

Instruction 

Intervention 

Professional 

Development 

Data 

Management 

System 

Diagnostic/ 

Achievement 

tests at each 

grade level 

School-Based 

Analysis of 

Student Work 

5 Block Model: 

Community Reading & 

Conversation 

Guided Reading 

Word Work 

Writing 

Self-Selected Reading 

Teacher Resource 

Materials, School and 

Classroom Materials 

Tier 1 

5 Block Literacy Model 

150-180 Minutes 

Tier 2 

Accommodations & 

Supplemental 

Materials (during 

Tier 1 instruction) 

Earobics 

Comprehension Plus 

Tier 3 

Reading Recovery 

Reading Mastery 

District Based 

Summer 

Institute 

School Based 

Sessions 




JCPS Professional 

Development Model 

TASK: Develop an Effective JCPS 

Framework that Links the Impact 

of PD to Student Achievement 

Issue 1: JCPS needs to standardize district-wide 

professional development 

Issue 2: Empirical connections between training 

and student achievement need to be 

established 




JCPS Professional 

Development Model 

E 

V 

A 

L 

U 

A 

T 

I 

O 

N 

Stage Stage 1: 1: 

ANALYSIS ANALYSIS 


DEVELOPMENT 

DEVELOPMENT 


DELIVERY DELIVERY 


IMPLEMENTATION 

IMPLEMENTATION 

Needs 

Objectives 

Format 

Evaluation Tools 

Instruction 

Piloting 

Operational Training 

-Participant Reaction 

-Participant Knowledge 

Prep for School-Based Evaluation 

Organization Support/Change 

Participant Use of New 

Knowledge and Skills 

Student Learning Outcomes 




Advantages of the JCPS 

PD Model 

• Requires Deliberate, Purposeful Training 

Development 

• Uses Proven Instructional Design Principles 

• Standardizes PD Presentations 

• Generates Meaningful Evaluation Data 

• Results in Reusable Training Support Packages 

(TSPs) 

• Embeds Quality Assurance Process 




Organizational Support 

for the JCPS PD Model 

Inter-Disciplinary PD Panel with Expertise in: 

•Evaluation 

•Professional Development 

•Curriculum 

•School Support 

•Literacy 

•Assessment 




E 

V 

A 

L 

U 

A 

T 

I 

O 

N 

JCPS Professional Development Process Flowchart 

Evidence 

insufficient 

Evidence 

insufficient 

Unfavorable 

evaluation 

results 

Content Expert/PD Panel Member conduct 

Stage 1: Analysis 

•Needs Assessment 

•Measurable Goals 

•Skills 

•Format 

PD System Committee(includes PD Panel 

member(s)) reviews analysis evidence 

Content Expert/PD Panel Member conduct 

Stage 2: Development 

•Measurable Objectives 

•Evaluation Tools 

•Instruction 

•Pilot TSP 

PD Panel reviews development evidence 

PD Presenter conducts Stage 3: Delivery 

• Operational delivery 

• Preparation for school based evaluation 

Content Experts review levels 1 and 2 

evaluation results 

PD Participant conducts 

Stage 4: Implementation 

Principal conducts Level 4 evaluation and with 

outside validation conducts levels 3 & 5 

Content Experts review level 3-5 

evaluation results 

Evidence 

sufficient 

Evidence 

sufficient 

Favorable 

evaluation 

results 




Major Outcomes of JCPS 

PD Model 

Stage 1: Analysis 

Multiple Data Sources Indicated an Implementation Issue with Teachers 

Delivering Guided Reading Instruction to Elementary Students. 

Stage 2: Development 

Created Professional Development Job Aid for Training Developers based 

on PD Model. 

Held Workshops with District Literacy Specialists who began TSP 

Development Process. 

Further Analyses Revealed that the Classroom Organizational Support 

Systems/Materials for Guided Reading Instruction were Inadequate. 

Selected a guided reading program compatible with the major tenets of 

the PD model 




Major Outcomes Continued 

Stage 3: Delivery 

1400 teachers attended guided reading institute. Training topics 

included assessment, guided reading, shared reading, classroom 

management, and explicit strategy instruction 

All training was standardized. Collected participant reaction and 

knowledge data 

Established plan to measure 5 levels of evaluation of guided 

reading professional development institute 




Five Levels of Professional 

Development Evaluation* 

Level 1 - Participants’ Reactions 

Level 2 - Participants’ Learning 

Level 3 - Organization Support & Change 

Level 4 - Participants’ Use of New 

Knowledge & Skills 

Level 5 - Student Learning Outcomes 

*Adapted from Guskey, T.R. (2002) Evaluating Professional 

Development. Thousand Oaks, CA: Corwin Press 




Level 1 Evaluation: 

Participants’ Reactions 

Purpose 

Evaluation 

Question 

Data Collection 

Method 

Outcome 

Implication 

To measure participants’ reactions/ satisfaction with the 

professional development experience 

Did the participants understand the training materials and 

find them useful? 

Questionnaires administered at the end of the guided 

reading PD institute 

The percentage of participants responded as follows: 

85% - extend content knowledge and professional practices 

95% - materials were appropriate 

93% - information was relevant to classroom 

94% - likely to what they have learned in their classroom 

Results used to improve program design and delivery of 

follow-up training sessions 





Participants’ Learning 

Purpose 

Evaluation 

Question 


Method 

Outcome 

Implication 

To measure participants’ new knowledge and skills 

Did the participants meet the presenters’ expected 

performance criteria (training objective)? 

Questionnaire; Product Demonstrations at end of PD 

session 

91.8% of participants were judged as meeting the 

expected performance criteria for the end of session 

Results used to improve program content, format, and 

organization 





Organization Support & Change 

Purpose 

Evaluation 

Question 


Method 

Implication 

To measure the organization’s s capacity, support and facilitation 

of the implementation of participant’s s new knowledge and skills 

Did the training affect organizational climate and procedures? 

Were sufficient resources made available? 

Was implementation facilitated and supported? 

Questionnaires of principal, instructional coaches, literacy lead 

teachers, literacy teachers 

High Ratings Low Ratings 

PD supports school goals Financial Support 

Quality follow-up support Encouraging PD through 

Access to expertise incentives & resources 

Sufficient leadership & Collaboration in planning PD 

collegial support 

Results used to document and improve organization support 





Participants’ Use of New Knowledge & Skills 

Purpose 

Evaluation 

Question 

Data 

Collection 

Method 

Outcome 

Implication 

To measure the degree and quality of implementation of 

participants’ use of new knowledge and skills 

Did participants effectively apply the new knowledge and 

skills? 

Classroom observations of guided reading lessons using 4 

point rubric conducted by instructional coaches twice a year 

and monthly by principals 

Baseline data: 

64% of primary level classrooms at full implementation 

51.3% of intermediate level classrooms at full implementation 

Results used to document and improve the implementation 





Student Learning Outcomes 

Purpose 

Evaluation 

Question 


Method 

Outcome 

Implication 

To measure the impact of PD on student achievement 

in reading 

What was the impact on student reading performance? 

Developmental Reading Assessment (administered 2 

times a year for all students, 3 times for struggling 

readers) 

Baseline: 48.1% of students met benchmark 

Current: 61.7% of students met benchmark 

Results used to improve teacher implementation and 

demonstrate the impact of teacher PD on student 

achievement 




Next Steps: 

• Level 1 and 2 evaluation data from the summer institute in 

guided reading will shape follow-up training during the school 

year 

• Level 3 data on organizational support from other key 

stakeholders will be collected and analyzed 

• Level 4 data on implementation will be collected in December 

and May 

• Level 5 student outcome data will continued to be analyzed 

during school year 

• PD panel will update and extend PD model 

• Evaluation of Guided Reading PD will be integrated with other 

evaluation efforts (implementation & outcome) of larger 

literacy initiative

Shaping the Future - Jefferson County Public Schools

Create successful ePaper yourself

Delete template?

Save as template?