Data Collection and Evaluation to Promote Student Learning

hfrp.org

Data Collection and Evaluation to Promote Student Learning

Data Collection and Evaluation

to Promote Student Learning

PIRC Annual Conference

Washington, DC

August 3, 2011

Ron Mirr Zena Rudo

RM Consulting

SEDL

Iowa City, IA Austin, TX

rmirr@mac.com

zrudo@sedl.org


Participants will learn...

ü Strategies to best showcase

PIRC accomplishments using

data

ü Linking with student-level

data

ü Action steps for using your

data to support family and

community engagement


Basics for Telling Your Story

• Have quality data to describe what you did.

• Be able to show how well you did it.

• Know what factors influenced what you did.

• Support the outcomes and

benefits of what you did with data.


“Collecting data is only the first step

toward wisdom, but sharing data is

the first step toward community.”

Henry Louis Gates Jr.

As used in the IBM/Linux Commercials


Basics for Telling Your Story

• Include information from each:

- Program delivery

- Stakeholder satisfaction

- Program outcomes

• Be concise and identify highlights.


Basics for Telling Your Story

• Use different perspectives to tell

your story.

• Use charts/graphs/photos/

vignettes.

• Grab the reader/listener with

some emotion.

• Tailor to your audience


Quality Data

• Accurate

• Useful

• Secure

• Timely

• Accessible

• Understandable

• Actionable

• Meaningful

When it is a process that is an integral

part of the learning environment.


Using Data to Tell Your Story

• Tell a

PIRC’s services

story about your

• Tell a

your PIRC has done

story about what

• Tell the

story about what

your PIRC has accomplished


Your Story Is Only As Good As

What You Collect

• What gets measured gets done.

• If you don’t measure results, you can’t tell

success from failure.

• If you can’t see success, you can’t reward it.

• If you don’t recognize failure, you can’t correct it.

• If you don’t find successes and failures, you can’t

learn and move forward.


What Did You Do

Good

Counts of who you served:

– Type (parent, teachers, administrators)

– Ethnicity

– Socio-economic status

– Location

Counts of the services you delivered:

– Type (workshops, newsletters, materials)

– Intensity (one time workshops, multi-session

programs)

Tip: Don’t inflate counts by duplicating service recipients.


How Well Did You Do It

Quality of services delivered

Good

– Trainer/participant ratio

– Participants report on quality

– Use of evidence-based materials

Satisfaction with services

– Participant report on satisfaction

Tip: Compare responses across participants by different

characteristics, (i.e., role, ethnicity, location, and child’s age).


What Not To Do With Data

Improved road conditions

Better-timed traffic signals

More highway lanes

Reduced construction delays

Improved signage

Source: USA Today.com. Reprinted by SEDL with permission of USA Today. All rights reserved.


What Factors Influenced Your Work

Better

Internal

– Characteristics – Behavior

– Communication – Attitude

External

– Environment – Culture

– Situation – History

Tip: Benchmark best practices to other successful programs.


Are Recipients of Your Services

Better Off

Changes that can happen more quickly

Better

– Knowledge – Skill level

– Awareness – Attitude

Changes that take longer to achieve

– Behavior

– Condition

Tip: Compare to similar groups not receiving services.


Are Students Better Off

Best

Establish a link between parent

engagement and student outcomes:

– Knowledge – Skill level

– Awareness – Attitude

– Behavior – Relationships

Tip: Emphasize 3-4 strong results; however, don’t use terms

that overstate the effect you made.


The Wyoming PIRC Story

Collect student data based on what is known about

student’s learning attributes from:

– Kathy Hoover-Dempsey & Howard Sandler’s Model of Parental

Involvement

– Nebraska PIRC & 21 st CCLC program study

– Nancy Hill’s Project PASS and Project Alliance

Handout is available at http://www.ronmirr.com on the downloads page.


Student Learning Attributes (Hoover-Dempsey)

Students believe they can learn

– I can do even the hardest homework if I try.

– I can learn the things taught in school.

– I can figure out difficult schoolwork.

Student want to learn because it’s important to them

and not driven by external rewards

– I want to understand how to solve problems.

– I like to look for more information about school subjects.

– I want to learn new things.

Hoover-Dempsey, K.V., et. al. (2005). Why do parents become involved Research findings and implications. Elementary

School Journal, 106(2), 105-130.

Kathy Hoover-Dempsey is currently at Vanderbilt University, Family-School Partnership Lab


Student Learning Attributes cont.

Students know how to manage their own learning

– I ask myself questions as I go along to make sure my homework makes sense

to me.

– I try to figure out the hard parts of my schoolwork on my own.

– I go back over things I don’t understand.

– I try to find a place that makes it easier to do my homework.

Students know how to ask for help, especially from teachers

– I can get along with most of my teachers.

– I can go and talk with most of my teachers.

– I can get my teachers to help me if I have problems with other students.

– I can explain what I think to most of my teachers.

– I ask the teacher to tell me how well I'm doing in class.


Parent-Child Conversations

(Nebraska PIRC & 21 st CCLC study)

Parent-child conversations that focus on learning

– I talk to my family about my homework

– I talk to my family about what I'm learning in school.


Aspirational Items (Nancy Hill)

Current and future expectations

– I plan to continue my education after high school.

– My family expects me to do well in school.

Hill, N. E., & Chao, R. K. (2009). Families, schools, and the adolescent: Connecting research, policy, and practice.

Nancy Hill is currently at the Harvard Graduate School of Education, Radcliffe Institute of Advanced Studies.


Sample Elementary & Middle School Scores

Item

Grade

5

Grades

6-­‐8

D

I plan to con3nue my educa3on a)er high school 97% 76% -­‐21%

I talk to my family about my homework 68% 48% -­‐20%

I talk to my family about what I'm learning in school. 77% 48% -­‐29%

I like to look for more informa3on about school subjects. 68% 48% -­‐20%

I go back over things I don’t understand. 87% 68% -­‐19%

I can explain what I think to most of my teachers. 78% 56% -­‐22%

I ask the teacher to tell me how well I'm doing in class. 61% 51% -­‐10%


How WY PIRC Uses Its Data

Intensive 2 year program (Solid Foundation)

– Baseline student data

– Use student data to prioritize areas of focus

– Monthly team meetings (parents and educators)

– Integrate data theme into all activities

• Compact

• Parent Nights

• Conferences

• Home Visits

– Reassess student data each year in the fall to make midcourse

corrections


The CA PIRC (CABE) Story

Project INSPIRE: Family-School Community

Leadership Development Program

Level 1- Awareness: Critical information impacting their

children (12 modules)

Level 2 – Mastery: In-depth understanding of critical

information impacting their children (12 modules)

Level 3 – Expert: Development and refinement of leadership

knowledge and skills (16 modules )


Project INSPIRE Research Study

Primary Research Questions:

1. Is the intensive parent leadership program

effective in helping parents increase the type,

frequency, and intensity of support they provide

their child’s learning

2. What impact, if any, does the type, frequency,

and/or intensity of parent engagement impact

student academic achievement


Students’ ELA Scores Increase

Schools In Need of Improvement

Non Treatment

Schools

increase

Treatment

School Increase

CA-PIRC

Increase

Total Increase

for Project

INSPIRE

Students

5.7 points

Additional 4.8

points

Additional 8.8

points

13.6 points


Students’ Math Scores Increase

Schools In Need of Improvement

Non Treatment

Schools

increase

Treatment

School Increase

CA-PIRC

Increase

Total Increase

for Project

INSPIRE

Students

4.9 points

Additional 9.9

points

Additional 17

points

26.9 points


Two Year Trend in Students’ ELA and

Math Achievement Gains

Parents Received Level 2 Services

CA Standards Test 2009` Increase 2010 Increase

English Language Arts 12.8 13.8

MathemaKcs 18.5 26.6


CA PIRC – CABE Moving the Field

Forward

What We Learned:

1. Knowledge is Power – parents need information to

participate fully in their children’s schooling.

2. Parents participate when they can work

collaboratively with the school(s).


CA PIRC – CABE Moving the Field

Forward

What We Learned:

3. Parents are more willing to engage in learning about

schooling when they are respected and valued as

partners.

4. Parental engagement does not happen nor is it

sustained when there is no shared common language

or structure at the school/district.


Your Data, Your Story

• Describe two accomplishments that make your

program high quality.

• Describe two accomplishments that make your

program successful.

• What data do you have to show you’ve made these

accomplishments

• What data challenges do you face to be able to

showcase these accomplishments


Done That – What’s Next

Which best describes your PIRC story

Haven’t yet been able to tell this story.

Told this story many times, but not with student data.

Told this story using student data.


Simple Rules for Getting Student-

Level Data

• Know exactly what you want

• Have a common understanding

• Set goals

• Know what data are available

• Collect multiple forms


Simple Rules for Getting Student-

Level Data

• Get public data first

• Find the right resource

• Be polite, but persistent

• Submit a written request

• Use your connections


Using Your Story to Move Forward

• Spread the word

• Start conversations

• Assist families to understand

• Ask questions

• Get involved

• Get additional funding


The Mechanics of Telling Your Story

• What data will be included and how

• What are the key messages your data support

• Who will you tell

• How will you tell your story


What Not To Do When Telling Your Story

Source: CartoonStock.com. Reprinted by SEDL with permission of CartoonStock.com. All rights reserved.


What Some May Say In Their Story

"IT HAS LONG BEEN KNOWN"...

I didn't look up the original reference.

"IT IS GENERALLY BELIEVED THAT"...

A couple of others think so, too.

"A CAREFUL ANALYSIS OF OBTAINABLE DATA"...

Three pages of notes were obliterated when I knocked over a beer glass.

"IT IS HOPED THAT THIS STUDY WILL STIMULATE FURTHER

INVESTIGATION IN THIS FIELD"...

I am pleased to feed you this rubbish.

Retrieved on July 27, 2011 from: http://www.guy-sports.com/humor/jokes/jokes_academic.htm


If I’m Going to Remember One

Thing From Today’s Session…

Using data to see what works

and what does not work in

achieving goals.

Data has no meaning on its

own. Meaning is a result of

human interaction with data.


Questions and Answers

More magazines by this user
Similar magazines