LISTENING

nemp.otago.ac.nz
  • No tags were found...

Listening and Viewing - NEMP - University of Otago

LISTENING AND VIEWING a s s e s s m e n t r e s u l t s 2002 nemp

Listening

and

Viewing

Assessment Results

2002

EARU

Lester Flockton

Terry Crooks

national education monitoring report 25


©2003 Ministry of Education, New Zealand

This report was prepared and published by The Educational Assessment Research Unit,

University of Otago, New Zealand, under contract to the Ministry of Education, New Zealand.

NATIONAL EDUCATION MONITORING Report 25

ISSN 1174-0000

ISBN 1–877182–39-7

NEMP REPORTS

1995 1 Science

2 Art

3 Graphs, Tables and Maps

1999 13 Science

14 Art

15 Graphs, Tables and Maps

16 Māori Students’ Results

Cycle 1

1996 4 Music

5 Aspects of Technology

6 Reading and Speaking

1997 7 Information Skills

8 Social Studies

9 Mathematics

Cycle 2

2000 17 Music

18 Aspects of Technology

19 Reading and Speaking

20 Māori Students’ Results

2001 21 Information Skills

22 Social Studies

23 Mathematics

24 Māori Students’ Results

1998 10 Listening and Viewing

11 Health and Physical Education

12 Writing

2002 25 Listening and Viewing

26 Health and Physical Education

27 Writing

28 Māori Students’ Results

Cycle 3

Forthcoming

2003 Science

Art

Graphs, Tables and Maps

Māori Students’ Results

EDUCATIONAL ASSESSMENT RESEARCH UNIT

PO Box 56, Dunedin, New Zealand Tel: 0800 808 561 Fax 64 3 479 7550

Email earu@otago.ac.nz

Website http://nemp.otago.ac.nz


LISTENING AND VIEWING 1

CONTENTS

Ac k n o w l e d g e m e n t s 2

Su m m a r y 3

Ch a p t e r 1 Th e Na t i o n a l Ed u c a t i o n Mo n i t o r i n g Pr o j e c t 5

Ch a p t e r 2 As s e s s i n g Li s t e n i n g a n d Viewing 9

Ch a p t e r 3 Li s t e n i n g 13

TREND TASKS:

Phone Message 14

The Clumsy Tiger 15

Cats Sleep Anywhere 16

Edmund Hillary Conquers

Mount Everest 17

RELEASED TASKS:

The Wind and the Sun 18

Weather Forecast 19

Drummer Dylan 20

Line Up 21

Zippos 22

Lexis 24

LINK TASKS: 1 – 8 25

Ch a p t e r 4 Viewing 26

TREND TASKS:

It’s Cool to Read 27

The Wolf 28

Santa Gets Ready 29

TV Commercials 30

Bedroom Plan 32

House Plan 33

RELEASED TASKS:

Māori Gods 34

Sweet Stall 35

Weet-Bix Card 36

Poster 37

Minties Moments 38

LINK TASKS: 9 – 16 39

Ch a p t e r 5 Pe r f o r m a n c e o f Su b g r o u p s 40

Ch a p t e r 6 Pacific Su b g r o u p s 42

Ap p e n d i x Th e Sa m p l e o f Sc h o o l s a n d St u d e n t s in 2002 44


2 ACKNOWLEDGEMENTS

The Project directors acknowledge the vital support and contributions of

many people to this report, including:

❑ the very dedicated staff of the Educational Assessment Research Unit

❑ Lisa Rodgers and other staff members of the Ministry of Education

❑ members of the Project’s National Advisory Committee

❑ members of the Project’s Literacy Advisory Panel

❑ principals and children of the schools where tasks were trialled

❑ principals, staff, and Board of Trustee members of the 283 schools included

in the 2002 sample

❑ the 3137 children who participated in the assessments and their parents

❑ the 107 teachers who administered the assessments to the children

❑ the 44 senior tertiary students who assisted with the marking process

❑ the 204 teachers who assisted with the marking of tasks early in 2002.


SUMMARY 3

New Zealand’s National Education Monitoring Project commenced

in 1993, with the task of assessing and reporting on the achievement

of New Zealand primary school children in all areas of the school

curriculum. Children are assessed at two class levels: year 4 (halfway

through primary education) and year 8 (at the end of primary

education). Different curriculum areas and skills are assessed each

year, over a four-year cycle. The main goal of national monitoring is

to provide detailed information about what children can do so that

patterns of performance can be recognised, successes celebrated, and

desirable changes to educational practices and resources identified

and implemented.

Each year, small random samples

of children are selected nationally,

t h e n a s s e s s e d i n t h e i r o w n

schools by teachers specially

seconded and trained for this

work. Task instructions are given

orally by teachers, through video

presentations, or in writing. Many

of the assessment tasks involve the

children in the use of equipment

and supplies. Their responses are

presented orally, by demonstration,

in writing, or through submission

of other physical products. Many

of the responses are recorded on

videotape for subsequent analysis.

In 2002, the fourth year of the

second cycle of national monitoring,

two areas were assessed: health and

physical education, and the writing,

listening and viewing components

of the English curriculum. This

report presents details and results

of the assessments of student skills

and knowledge in listening and

viewing.

Many of the tasks were used with

both year 4 and year 8 students,

which allows direct comparisons

of the performance of year 4 and 8

students in 2002. Because some of

the tasks were used both in 1998

and in 2002, trends in performance

across the four-year period can also

be examined.

ASSESSING LISTENING AND VIEWING

Chapter 2 explains the place of

listening and viewing in the New

Zealand curriculum and presents

the frameworks

for listening and

v i e w i n g . T h e

listening framework

has as its central

organising theme

constructing

meaning from spoken messages

and communications for a range

of purposes. Seven purposes are

specified in the framework, together

with a number of understandings,

skills and attitudes that students

and their teachers are working to

develop. The viewing framework has

as its theme constructing meaning

from visual texts. In other respects

it has a parallel structure to the

listening framework.

LISTENING

Chapter 3 presents results from

the tasks that assessed the students’

listening skills. Averaged across 141

task components administered to

both year 4 and year 8 students,

11 percent more year 8 than year

4 students succeeded with these

components. The trend analyses

showed almost no change since

1998. Averaged across 25 task

components attempted by year 4

students in both years, the same percentage of students succeeded in 2002 as

in 1998. At year 8 level, with 24 task components included in the analysis, on

average 1 percent more students succeeded in 2002 than in 1998. Students

generally achieved quite high performance levels on task

components that involved recalling and using specific

factual information. Predictably, they were less successful

where the task components involved interpretation or

inference, such as distinguishing facts from opinions,

interpreting messages in a story, or evaluating the merits

of opposing arguments. This performance contrast was

particularly evident for year 4 students.


4 NEMP Report 25: Listening and Viewing 2002

WRITING

Chapter 4 presents results for

the viewing tasks, which assessed

the students’ capabilities in

constructing meaning from visual

material. Averaged across 129 task

components administered to both

year 4 and year 8 students, 9 percent

more year 8 than year 4 students

succeeded with these components.

The trend analyses showed small

gains since 1998. Averaged across

21 task components attempted

by year 4 students in both years, 2

percent more students succeeded in

2002 than in 1998. These gains are

not large enough to be regarded as

significant. At year 8 level, with 29

task components included in the

analysis, 3 percent more students

succeeded in 2002 than in 1998,

representing a modest gain. Both year

4 and year 8 students often achieved

quite high performance levels on task

components that involved observing,

recalling and using specific factual

information, but were less successful

where the task components involved

interpretation or evaluation of visual

messages, or of the intentions of the

designers of those messages. These

latter components usually were

substantially better handled by year

8 than year 4 students.

PACIFIC SUBGROUPS

Chapter 6 reports the results of

analyses of the achievement of

Pacific Island students.

Additional sampling

o f s c h o o l s w i t h

high proportions of

Pacific Island students

permitted comparison

of the achievement of

Pacific Island, Māori

PERFORMANCE OF SUBGROUPS

Chapter 5 reports the results

o f a n a l y s e s t h a t c o m p a r e d

the performance of different

demographic subgroups. School

size, school type (full primary or

intermediate) and community

size did not seem to be important

factors predicting achievement on

listening and viewing tasks. South

Island students performed better

than Auckland students on about 40

percent of the listening and viewing

tasks at year 4 level, but only about 5

percent of the year 8 tasks. At both

year levels, girls performed better

than boys on some tasks, with the

proportion of these tasks increasing

somewhat from year 4 to year 8

(14 to 29 percent for listening, 6

to 11 percent for viewing). Non-

Māori students outperformed Māori

students on about 35 percent of the

viewing tasks at both year levels and

of the listening tasks at year 4 level,

but this dropped to 18 percent of the

year 8 listening tasks. The SES index

based on school deciles showed the

strongest pattern of differences,

with differences on 50 to 70 percent

of listening and viewing tasks at both

year levels.

Between 1998 and 2002, there

have been noteworthy changes in

subgroup differences for four of the

seven variables. The only variable

showing increased disparity was

and other children attending schools

that have more than 15 percent

Pacific Island students enrolled. The

results apply only to such schools,

but it should be noted that about 75

percent of all Pacific students attend

schools in this category.

Compared to Māori and “other”

students in these schools with more

than 15 percent Pacific students,

geographic zone, and that only at

year 4 level, with the performance

gap between South Island and

Auckland students increasing

between 1998 and 2002 (from 13%

to 36% of listening tasks, and from

22% to 44% of viewing tasks). On the

other hand, there were substantial

reductions in subgroup differences

for three variables: gender, ethnicity

and the SES index based on school

deciles. Over the four year period,

the percentage of viewing tasks on

which girls performed better than

boys decreased from 22 percent to

6 percent for year 4 students and

from 29 percent to 11 percent for

year 8 students. The percentage

of tasks on which Māori students

scored lower than other students

decreased substantially for listening

and viewing tasks at both year levels

(50% to 36% for year 4 listening tasks,

33% to 18% for year 8 listening tasks,

67% to 38% for year 4 viewing tasks,

and 57% to 33% for year 8 viewing

tasks). Similarly, the percentage of

tasks on which students from low

decile schools scored significantly

lower than students from high decile

schools decreased for both sets of

tasks at both year levels (87% to 71%

for year 4 listening tasks, 78% to 59%

for year 8 listening tasks, 100% to 50%

for year 4 viewing tasks, and 86% to

61% for year 8 viewing tasks).

year 4 Pacific students performed

less well than the “other” students

on 37 percent of the tasks but

similarly to Māori students. In the

corresponding results at year 8

level, the Pacific students performed

less well than the “other” students

on 24 percent of the tasks and less

well than the Māori students on

11 percent of tasks.


CHAPTER 1 5

THE NATIONAL EDUCATION MONITORING PROJECT

This chapter presents a concise outline of the rationale

and operating procedures for national monitoring,

together with some information about the reactions

of participants in the 2002 assessments. Detailed

information about the sample of students and schools is

available in the appendix (p44).

Purpose of national monitoring

The New Zealand Curriculum Framework (1993,

p26) states that the purpose of national monitoring

is to provide information on how well overall

national standards are being maintained, and where

improvements might be needed.

The focus of the National Education Monitoring

Project (NEMP) is on the educational achievements

and attitudes of New Zealand primary and intermediate

school children. NEMP provides a national “snapshot” of

children’s knowledge, skills and motivation, and a way to

identify which aspects are improving, staying constant,

or declining. This information allows successes to be

celebrated and priorities for curriculum change and

teacher development to be debated more effectively,

with the goal of helping to improve the education which

children receive.

Assessment and reporting procedures are designed to

provide a rich picture of what children can do and thus to

optimise value to the educational community. The result

is a detailed national picture of student achievement. It

is neither feasible nor appropriate, given the purpose

and the approach used, to release information about

individual students or schools.

Monitoring at two class levels

National monitoring assesses and reports what

children know and can do at two levels in primary and

intermediate schools: year 4 (ages 8-9) and year 8 (ages

12-13).

National samples of students

National monitoring information is gathered using

carefully selected random samples of students, rather

than all year 4 and year 8 students. This enables a

relatively extensive exploration of students’ achievement,

far more detailed than would be possible if all students

were to be assessed. The main national samples of 1440

year 4 children and 1440 year 8 children represent about

2.5 percent of the children at those levels in New Zealand

schools, large enough samples to give a trustworthy

national picture. Additional samples of 120 children at

each level allow the achievement of Pacific students to

be assessed and reported. At year 8 level only, a special

sample of 120 children learning in Māori immersion

schools or classes is selected. Their achievement can

then be compared with the achievement of Māori

students in the main year 8 sample, whose education

is predominantly in English (these comparisons are not

reported here, but in a separate report that shows the

tasks in both Māori and English).

Three sets of tasks at each level

So that a considerable amount of information can be gathered

without placing too many demands on individual

students, different students attempt different tasks. The

1440 students selected in the main sample at each year

level are divided into three groups of 480 students, comprising

four students from each of 120 schools.

Timing of assessments

The assessments take place in the second half of the

school year, between August and November. The year 8

assessments occur first, over a five-week period. The year

4 assessments follow, over a similar period. Each student

participates in about four hours of assessment activities

spread over one week.

Specially trained teacher administrators

The assessments are conducted by experienced teachers,

usually working in their own region of New Zealand.

They are selected from a national pool of applicants,

attend a week of specialist training in Wellington

led by senior Project staff, and then work in pairs to

conduct assessments of 60 children over five weeks.

Their employing school is fully funded by the Project to

employ a relief teacher during their secondment.


6 NEMP Report 25: Listening and Viewing 2002

YEAR

NEW ZEALAND CURRICULUM

2003

Science

1

2

3

4

(1999)

(1995)

2004

(2000)

(1996)

2005

(2001)

(1997)

2006

(2002)

Art

Social Studies: graphics, tables, maps, charts & diagrams

Language: reading and speaking

Aspects of Technology

Music

Mathematics: numeracy skills

Social Studies

Information Skills: library, research

Language:

writing,

listening, viewing

Communication skills

Problem-solving skills

Self-management and competitive skills

Social and co-operative skills

Work and study skills

Attitudes

(1998)

Health and Physical Education

Four-year assessment cycle

Each year, the assessments cover about one quarter of

the national curriculum for primary schools. The New

Zealand Curriculum Framework is the blueprint for the

school curriculum. It places emphasis on seven essential

learning areas, eight essential skills, and a variety of

attitudes and values. National monitoring aims to address

all of these areas, rather than restrict itself to preselected

priority areas.

The first four-year cycle of assessments began in 1995

and was completed in 1998. The second cycle ran from

1999 to 2002. The third cycle begins in 2003 and will

finish in 2006. The areas covered each year and the

reports produced for cycle 2 are listed inside the front

cover of this report.

Some of the tasks are kept constant from one cycle to the

next. This re-use of tasks allows trends in achievement

across a four-year interval to be observed and reported.

Starting in 2002, the percentage of tasks retained is

increasing from 35 to 45 percent, so that trends can be

reported more thoroughly.

Important learning outcomes assessed

The assessment tasks emphasise aspects of the

curriculum which are particularly important to life in

our community, and which are likely to be of enduring

importance to students. Care is taken to achieve

balanced coverage of important skills, knowledge and

understandings within the various curriculum strands,

but without attempting to slavishly follow the finer

details of current curriculum statements. Such details

change from time to time, whereas national monitoring

needs to take a long-term perspective if it is to achieve

its goals.

Wide range of task difficulty

National monitoring aims to show what students know

and can do. Because children at any particular class level

vary greatly in educational development, tasks spanning

multiple levels of the curriculum need to be included if

all children are to enjoy some success and all children

are to experience some challenge. Many tasks include

several aspects, progressing from aspects most children

can handle well to aspects that are less straightforward.

Engaging task approaches

Special care is taken to use tasks and approaches that

interest students and stimulate them to do their best.

Students’ individual efforts are not reported and have

no obvious consequences for them. This means that

worthwhile and engaging tasks are needed to ensure

that students’ results represent their capabilities rather

than their level of motivation. One helpful factor is that

extensive use is made of equipment and supplies which

allow students to be involved in “hands-on” activities.

Presenting some of the tasks on video or computer

also allows the use of richer stimulus material, and

standardizes the presentation of those tasks.

Positive student reactions to tasks

At the conclusion of each assessment session, students

completed evaluation forms in which they identified

tasks that they particularly enjoyed and tasks that did not

appeal, using a three-point rating scale. Averaged across

all tasks in the 2002 assessments, 73 percent of year 4

students indicated that they particularly enjoyed the

tasks. The range across the 112 tasks was from 97 percent

down to 52 percent. As usual, year 8 students were more

demanding. On average, 62 percent of them indicated

that they particularly enjoyed the tasks, with a range


Chapter 1: The National Education Monitoring Project 7

across 133 tasks from 94 percent down to 35 percent. No

task was more disliked than liked. The students’ parents

and teachers also reacted very positively to the tasks and

assessment approaches.

Appropriate support for students

A key goal in Project planning is to minimise the extent

to which student strengths or weaknesses in one area

of the curriculum might unduly influence their assessed

performance in other areas. For instance, skills in reading

and writing often play a key role in success or failure in

paper-and-pencil test areas such as science, social studies,

or even mathematics. In national monitoring, a majority

of tasks are presented orally by teachers, on videotape,

or on computer, and most answers are given orally or by

demonstration rather than in writing. Where reading or

writing skills are required to perform tasks in areas other

than reading and writing, teachers are happy to help

students to understand these tasks or to communicate

their responses. Teachers are working with no more

than four students at a time, so are readily available to

help individuals.

To free teachers further to concentrate on providing

appropriate guidance and help to students, so that the

students achieve their best efforts, teachers are not

asked to record judgements on the work the students

are doing. All marking and analysis is done later, when

the students’ work has reached the Project office in

Dunedin. Some of the work comes on paper, but much

of it arrives recorded on videotape. In 2002, about half

of the students’ work came in that form, on a total of

about 4800 videotapes. The video recordings give a

detailed picture of what students and teachers did and

said, allowing rich analysis of both process and task

achievement.

Four-task approaches used

In 2002, four task approaches were used. Each student

was expected to spend about an hour working in each

format. The four approaches were:

◗ One-to-one interview. Each student worked individually

with a teacher, with the whole session

recorded on videotape.

◗ Stations. Four students, working independently,

moved around a series of stations where tasks had

been set up. This session was not videotaped.

◗ Team and Independent. Four students worked

collaboratively, supervised by a teacher, on some

tasks. This was recorded on videotape. The students

then worked individually on some paper-and-pencil

tasks.

◗ Open Space. Four students, supervised by two teachers,

attempted a series of physical skills tasks, with the

whole session recorded on videotape.

Professional development benefits for teacher

administrators

The teacher administrators reported that they found

their training and assessment work very stimulating

and professionally enriching. Working so closely with

interesting tasks administered to 60 children in at least

five schools offered valuable insights. Some teachers

have reported major changes in their teaching and

assessment practices as a result of their experiences

working with the Project. Given that 107 teachers served

as teacher administrators in 2002, or about half a percent

of all primary teachers, the Project is making a major

contribution to the professional development of teachers

in assessment knowledge and skills.

This contribution will steadily grow,

since preference for appointment

each year is given to teachers who

have not previously served as teacher

administrators. The total after seven

years is 782 different teachers.

Marking arrangements

The marking and analysis of the students’ work occurs

in Dunedin. The marking process includes extensive

discussion of initial examples and careful checks of the

consistency of marking by different markers.

Tasks which can be marked objectively or with

modest amounts of professional experience usually are

marked by senior tertiary students, most of whom have

completed two or three years of pre-service preparation

for primary school teaching. Forty-four student markers

worked on the 2001 tasks, employed 5 hours per day for

5 weeks.

The tasks that require higher levels of professional

judgement are marked by teachers, selected from

throughout New Zealand. In 2002, 204 teachers

were appointed as markers. Most teachers worked

either mornings or afternoons for one week. Teacher

professional development through participation in the

marking process is another substantial benefit from

national monitoring. In evaluations of their experiences

on a four-point scale (“dissatisfied” to “highly satisfied”),

76 to 94 percent of the teachers who marked student

work in 2002 chose “highly satisfied” in response to

questions about:

◗ the extent to which marking was professionally

satisfying and interesting;

◗ its contribution to professional development in the

area of assessment;

◗ whether they would recommend NEMP marking

work to colleagues;

◗ whether they would be happy to do NEMP marking

again.


10 NEMP Report 25: Listening and Viewing 2002

nemp listening framework

CENTRAL ORGANISING THEME

Constructing meaning from spoken messages and communications for a range of purposes

UNDERSTANDINGS

• Listening, speaking and thinking are interactive and

interdependent.

• Listening can be passive or active.

• Comprehension of spoken messages is affected by the interests,

purposes and background of the listener.

• Listeners are expected to follow social conventions which vary

according to context.

• Different cultures have different conventions and expectations.

• Listening involves recognition and interpretation of non-verbal

messages that accompany verbal communications.

• Active listening requires the listener to organise, analyse and

relate content to previous knowledge.

PURPOSES

• Participating in conversation.

• Following a story.

• Obtaining information.

• Identifying opinions and viewpoints.

• Critical evaluation.

• Enjoyment and inspiration.

• Acquiring new language and understandings.

SKILLS

• Attending and concentrating.

• Recalling and retelling what others have said.

• Comprehending literal meaning.

• Identifying main ideas or themes.

• Summarising.

• Thinking critically.

• Distinguishing fact from opinion; recognising bias and

prejudice.

• Making inferences.

• Drawing appropriate conclusions.

• Gauging mood and occasion.

• Knowing how and when to respond.

• Listening with empathy.

• Reading body language (smiles, nods, pauses).

• Exploring language and multiple meanings of messages.

• Relating unfamiliar words and phrases to context to derive

meaning.

MOTIVATION

• Enjoyment from listening to a variety of sources.

• Awareness of the benefits of listening.

• Commitment to being a good listener.

nemp viewing framework

CENTRAL ORGANISING THEME

Constructing meaning from visual texts

UNDERSTANDINGS

• Viewing is a complex thinking process which involves the

integration of information from many sources.

• Visual messages are created to inform, persuade and entertain.

• Different messages and meanings can be drawn from a visual

text.

• Responses to visual information can be critical or passive.

• Particular effects can be created by combining visual, aural and

verbal elements.

• Visual effects are used to appeal to different moods, feelings,

occasions and settings.

PURPOSES

• Following a story.

• Obtaining information.

• Identifying opinions and viewpoints.

• Critical evaluation.

• Enjoyment and inspiration.

• Acquiring visual language and understandings.

SKILLS

• Comprehending literal meaning.

• Interpreting symbolic elements.

• Recognising the interaction between words and images.

• Comparing written and visual versions of texts.

• Thinking critically about the intentions, effects and impact of

visual messages.

• Identifying and analysing the techniques and conventions of

visual language in a variety of contexts.

• Exploring ideas and multiple meanings.

• Recognising how visual texts are tailored to appeal to particular

audiences.

• Reading body language.

MOTIVATION

• Enthusiasm for viewing and responding to a wide variety of

visual information.

• Voluntary engagement with visual language.

• Commitment to exploring the meanings of visual messages.


12 NEMP Report 25: Listening and Viewing 2002

Trend tasks

Twenty-seven of the

thirty-seven tasks were

the same for both year

4 and year 8. One task

followed the same

administration and

marking procedures for

both year 4 and year 8,

but included fewer

components for year

4 students. Two tasks

were administered

only to year 4 students,

and seven tasks only to

year 8 students.

Ten of the tasks were used previously, entirely or in part,

in the 1998 listening and viewing assessments. These

were called link tasks in the 1998 report, but were not

described in detail to avoid any distortions in the 2002

results that might have occurred if the tasks had been

widely available for use in schools since 1998. In the

current report, these tasks are called trend tasks, and are

used to examine trends in student performance: whether

they have improved, stayed constant or declined over the

four-year period since the 1998 assessments.

Link tasks

To allow similar comparisons between the 2002 and

2006 assessments, 16 of the tasks used for the first time in

2002 have been designated link tasks. Results of student

performance on these tasks are presented in this report,

but the tasks are described only in general terms because

they will be used again in 2006.

Marking methods

The students’ responses were assessed using

specially designed marking procedures. The

marking criteria used had been developed in

advance by Project staff, but were sometimes

modified as a result of issues raised during

the marking. Tasks that required marker

judgement and were common to year 4 and

year 8 were intermingled during marking

sessions, with the goal of ensuring that the

same scoring standards and procedures were

used for both. Similarly, where the marking

of trend tasks required substantial marker

judgement, specially selected representative

samples of the 1998 performances were

re-marked, intermingled with the 2002

performances. This helped to ensure that

the trend information would be trustworthy,

unaffected by changes in marking standards

between 1998 and 2002.

Task by task reporting

National monitoring assessment is reported task by task

so that results can be understood in relation to what the

students were asked to do.

Access tasks

Teachers and principals have expressed

considerable interest in access to NEMP task

materials and marking instructions, so that

they can use them within their own schools.

Some are interested in comparing the performance of

their own students to national results on some aspects of

the curriculum, while others want to use tasks as models

of good practice. Some would like to modify tasks to

suit their own purposes, while others want to follow

the original procedures as closely as possible. There is

obvious merit in making available carefully developed

tasks that are seen to be highly valid and useful for

assessing student learning.

Some of the tasks in this report cannot be made

available in this way. Link tasks must be saved for use

in four years’ time, and other tasks use copyright or

expensive resources that cannot be duplicated by NEMP

and provided economically to schools. There are also

limitations on how precisely a school’s administration

and marking of tasks can mirror the ways that they are

administered and marked by the Project. Nevertheless,

a substantial number of tasks are suitable to duplicate

for teachers and schools. In this report, these access

tasks are identified with the symbol above, and can be

purchased in a kit from the New Zealand Council for

Educational Research (P.O. Box 3237, Wellington 6000,

New Zealand).

Teachers are also encouraged to use the NEMP web site

(http://nemp.otago.ac.nz) to view video clips and listen

to audio material associated with some of the tasks.


Chapter 3 13

listeninG

The assessments included eighteen tasks which asked the

students to listen to information presented orally or both orally

and visually, and to repeat the information, answer questions

using the information, or follow oral instructions. Some of

the recordings used in these tasks included pictures as well

as sound, but the details that students needed were provided

mainly on the soundtrack. Students need to be able to listen to

factual presentations, assertions, arguments or instructions, and

to recall, interpret or follow them correctly.

Thirteen tasks were identical for year 4 and year 8 students,

one was administered only to year 4 students, and four only to

year 8 students. Four are trend tasks (fully described with data

for both 1998 and 2002), six are released tasks (fully described

with data for 2002 only), and eight are link tasks (to be used

again in 2006, so only partially described here).

The tasks are presented in the three sections: trend tasks, then

released tasks and finally link tasks. Within each section, tasks

administered to both year 4 and year 8 students are presented

first, followed by tasks administered only to year 4 students and

then tasks administered only to year 8 students.

Averaged across 141 task components administered to both

year 4 and year 8 students, 11 percent more year 8 than

year 4 students succeeded with these components. Year 8

students performed better on 135 of the 141 components. The

components with the largest differences were scattered across

the tasks, as were the components on which year 8 students

did not do better than year 4 students.

The trend analyses showed almost no change since 1998.

Averaged across 25 task components attempted by year

4 students in both years, the same percentage of students

succeeded in 2002 as in 1998. Gains occurred on 9 components

and losses on 11 components. At year 8 level, with 24 task

components included in the analysis, 1 percent more students

succeeded in 2002 than in 1998. Gains occurred on 14

components, with losses on 10 components.

The students generally achieved quite high performance

levels on task components that involved recalling and using

specific factual information. Predictably, they were less

successful where the task components involved interpretation

or inference, such as distinguishing facts from opinions,

interpreting messages in a story, or evaluating the merits of

opposing arguments.


14 NEMP Report 25: Listening and Viewing 2002

Phone Message

trend task

Approach: One to one Level: Year 4 and year 8

Focus: Accurately recalling a message.

Resources: Audio recording on laptop computer.

Questions/instructions:

[This activity uses the computer for a soundtrack only. There are no visuals

provided on the computer.]

This activity uses the computer.

Click the Phone Message button.

On the video you are going to hear a message left on

an answerphone.

You will only hear the message once, so listen

carefully.

After you have listened to the message, I will ask you

to tell me what it said.

Put the headphones on so that I can’t hear.

Click the Play button.

Audio script:

Hello. This is Frank Kane of the town library speaking. I’d like

to leave a message for one of the teachers called Mrs Bright.

Would you tell her that it will be all right for her class to visit the

library next Tuesday at 11 o’clock. We’ll have the videos about

wild animals that she wants the kids to see. It takes about half

an hour. Thanks.

% responses

2002 (’98) 2002 (’98)

year 4 year 8

from Frank Kane

(or interpretable variant) 28 (22) 58 (51)

of town library

(or interpretable variant) 37 (26) 65 (60)

to Mrs Bright (or similar) 40 (37) 48 (54)

okay for class to visit library 78 (75) 82 (84)

next Tuesday 31 (25) 33 (36)

at 11am 32 (27) 47 (45)

has the video 62 (70) 77 (86)

about wild animals 58 (63) 66 (73)

video takes about half an hour 36 (37) 57 (54)

You’ve listened to the message and I didn’t hear it.

I want you to tell me what the message said.

Try to tell me all of the information.

Total score: 8–9 2 (1) 11 (14)

6–7 21 (18) 37 (36)

4–5 39 (39) 36 (34)

2–3 28 (32) 14 (14)

0–1 10 (10) 2 (2)

Commentary:

About 25 percent more year 8 than year 4 students

included 6 or more correct items of information in their

retelling. There was little change between 1998 and 2002

at either year level.


16 NEMP Report 25: Listening and Viewing 2002

Cats Sleep Anywhere

trend task

Approach: Station Level: Year 4

Focus: Following instructions.

Resources: Audio recording on laptop computer.

Questions/instructions:

This activity uses the computer.

Click on the button that says Cats Sleep Anywhere.

Click the Play button to hear the video.

Only play the tape once.

Audio script:

You have a picture of a room in a house and a little stamp

that makes a picture of a cat. You can practise using the

stamp. Make 3 stamps of the cat inside the Practice Box.

Do that now.

Allow time.

% responses

2002 (’98)

year 4

Places stamped: table 91 (91)

Put the stamp down now and listen to me.

You are going to listen to a poem called ‘Cats Sleep

Anywhere’. Look at the picture of the room. The poem tells

you the places where a cat might sleep. As the poem is read

stamp a picture of the cat in the different places where it says

the cat might sleep. Only stamp the places where I tell you.

Get your stamp ready now.

I’ll start the poem now.

Cats sleep anywhere

Any table, any chair

Top of piano, window ledge

In the middle, on the edge

Open drawer, empty shoe

Anybody’s lap will do

Fitted in a cardboard box

In a cupboard with your frocks

Anywhere, they don’t care

Cats sleep anywhere

chair 93 (92)

top of piano 97 (96)

window ledge 82 (94)

open drawer 93 (91)

empty shoe, boot or both 94 (92)

child’s lap 93 (93)

cardboard box in cupboard 94 (95)

Total score: 8 69 (76)

7 19 (12)

6 5 (5)

4–5 4 (4)

0–3 3 (3)

Commentary:

The year 4 students were very successful with this task. Slightly fewer students got a perfect score in 2002 than in

1998. This was largely related to lower success in placing a stamp on a ‘window ledge’. The marking of this

component may have been stricter in 2002.


18 NEMP Report 25: Listening and Viewing 2002

The Wind and the Sun

Approach: One to one Level: Year 4 and year 8

Focus: Retelling and critical thinking.

Resources: Video recording on laptop computer; 4 pictures.

Questions/instructions:

This activity uses the computer.

Click the The Wind and the Sun button.

% responses

y4 y8

We will start this activity by listening to a fable called

‘The Wind and the Sun’. A fable is a story which has a

moral or a lesson in it. You will need to listen carefully

because when the story has finished, I’m going to ask

you to tell it to me.

Click the Play button.

Video script:

One day, the Wind and the Sun were having an argument.

“I’m stronger than you!” said the Sun.

“No you’re not,” said the Wind. “I’m much stronger than you!”

As they argued, they saw a man wearing a red cloak, walking along

the road.

“I propose a test,” said the Wind. “Whichever one of us can tear the

cloak from the back of that man is the stronger.”

The Wind took the first turn. He blew so hard that leaves and branches

were torn from trees. Clothes hung out to dry were blown off the line.

Everyone hurried indoors for shelter.

But the Wind could not take the cloak off the man. The man shivered

and held tightly to his cloak.

“You have failed,” said the Sun. “And now it is my turn.”

The Sun shone. It was very warm. The man felt hot and thirsty.

He stopped to drink a cup of water.

The Sun shone brighter and the man became hotter and hotter.

He pulled off his boots. It was much too hot to wear them.

At last he came to a stream. He sat on the bank and dipped his feet in

the cool water. But the Sun shone warmly on his back.

The man decided to lie down in the shade of a tree to rest. He took off

his cloak and laid it across the grass.

“I have won,” said the Sun to the Wind.

“As you can see it is easier to influence people with gentleness than

with force.”

Here are some pictures that show parts of the story.

Place pictures in front of student in order 1 to 4.

1. Now I want you to retell the fable to me. Use these

pictures to help you as you tell the story to me.

Give student time to retell the fable.

There is a main message or lesson at the end of this

story which said: ‘it is easier to influence people with

gentleness than with force’.

2. Try to explain to me what that lesson means.

prompt: What does the main message

of the story mean?

2

1

4

3

% responses

% y4 responses y8

y4 y8

Details included: argument 83 92

between wind and sun 84 91

about who is stronger 71 88

test (who can get cloak off) 77 90

red cloak 24 37

wind blew 73 86

leaves/branches torn off trees 37 42

clothes blown off line 25 29

people sheltered 26 28

wind made man shiver/cold 15 23

man held on tightly to cloak 34 41

cloak stayed on 31 44

sun shone 59 75

man hot 62 73

man thirsty/drank water 11 12

man took off boots 78 84

put feet in water/stream 69 70

still sun shone on man’s back 31 40

wanted to rest/lie down 71 77

took off cloak 89 96

ending such as sun saying

“I have won” 50 51

Order of main events:

all in correct order 34 51

slightly jumbled 47 41

seriously jumbled 19 8

Explanation of message: very good 4 21

moderately good 29 42

poor 67 37

Total score: 19–21 4 10

16–18 19 27

13–15 30 34

10–12 21 19

7–9 16 7

less than 7 10 3

Commentary:

Both year 4 and year 8 students gave a fuller account

of the effects of the sun than of the effects of the wind.

About 15 percent more year 8 than year 4 students

obtained total scores of 16 or more.


Chapter 3: Listening 19

Weather Forecast

Approach: One to one Level: Year 4 and year 8

Focus: Inferential listening.

Resources: Audio recording on laptop computer.

Questions/instructions:

This activity uses the computer.

Click the Weather Forecast button.

Imagine your sports team wants to go to Wellington.

You are going to play outdoors for the whole day.

You want fine, dry and calm weather to do this.

The weather forecast you will hear tells you the

weather for a week. Listen to the forecast so you

can tell me what day you would choose to go to

Wellington. Remember, you want weather that is

fine, dry and calm.

Click the Play button.

[stationary image only on screen throughout task]

Audio script:

And now the 5-day forecast for the Wellington region.

Monday:

Strong westerlies over the Tararua Ranges with some

snow flurries over 800 metres.

Some light rain with skies clearing later in the day.

Tuesday:

Light rain forecast for the morning, with continued

drizzle expected throughout the day, lifting by early

evening.

Calm weather with little wind.

Wednesday:

Clear, calm weather expected throughout the region.

Warm and sunny in all areas. Temperatures dropping

at night-time with frost in some areas.

Thursday:

Winds rising in the morning. Mostly fine, however,

a few cloudy areas expected night and morning with

rain in the east.

Friday:

Continued wind with driving rain spreading

throughout the region.

1. Which day would you choose?

2. Why would you choose that day?

% responses

y4 y8

Wednesday 87 95

lack of rain 32 37

lack of wind 49 65

clear/sunny/frosty 83 85

Total score: 4 16 22

3 40 46

2 30 26

1 6 4

0 8 2

Commentary:

The task refers to wanting ‘fine, dry and calm weather’. While students generally chose the best day, they often

did not fully address these three conditions in their explanations. In particular, many did not address the issue

of wind.


20 NEMP Report 25: Listening and Viewing 2002

Drummer Dylan

Approach: One to one Level: Year 4 and year 8

Focus: Comprehension of news clip, thinking critically.

Resources: Video recording on laptop computer.

Video script:

Reporter: Dylan Elise has been beating his way to the top since

he was a little boy.

Dylan: My Dad had a drum kit and, he just let me have a go

on them. Well actually I hopped onto the drums and

then I just started playing away.

Reporter: How old were you?

Dylan: I was 6 at that point.

Mum: He was hopeless when he was 6, it was awful listening

to him, it was really painful.

Reporter: Yet, within a year Dylan was on stage, heard but not seen.

But now Dylan is older, wiser, he’s eleven.

Dylan; It took years of like training and practice.

Reporter: Today the kid from Tawa is trained by University tutors.

Tutor: He’ll be putting me out of a job.

Reporter: As good as graduates 2, 3 times his age.

Tutor:

It’s one of those things that’s a mystery, you can’t explain

it, you know. But, yeah he’s definitely got it.

Reporter: Eleven years old and he’s snaring guest spots with the

Youth and New Zealand Symphony Orchestras.

Questions/instructions:

This activity uses the computer.

Click the Drummer Dylan button.

We’re going to watch a video from the news

about an 11 year old boy who plays the

drums. Listen carefully, and I’ll ask you some

questions when the video finishes.

Click the Play button.

1. How did Dylan get started on the drums?

% responses

y4 y8

used Dad’s drum kit (hopped onto the drums) 56 82

2. How old was Dylan when he began

playing the drums? 6 years old 94 97

3. How much practice does Dylan put in?

3 hours a day 60 74

usually every day 18 26

4. Why did Dylan say it’s important to keep fit?

uses lots of muscles a lot 14 50

Dylan: It was a great honour just being in the same room as

members of the Symphony Orchestra, but playing with them

was, an even bigger thrill.

It’s important to keep fit, because you have to use the quads,

calves for like hitting the base drums and high hat and then

like the shoulders, biceps, triceps.

I practise 3 hours a day, and that’s normally every day.

Reporter: But no playing after 5pm, an imposed neighbourhood

curfew.

Dylan: Neighbours shouting from over the fence.

Reporter: Shouting what?

Dylan: Oh, shut those drums up.

Reporter: And, Dylan is drumming up a storm elsewhere. He’s the 11

year old North Island champion for the 100 and 200 metres,

National champion for the 400. Yet he’s still a normal

kiwi kid.

Dylan: I hope I can get into the All Blacks, that’s what one of my

dreams are.

Reporter: Of course it is. Until then, the world is at his finger tips.

5. Why doesn’t Dylan practise after 5 o’clock?

% responses

y4 y8

neighbours complain/curfew 54 86

6. What did the reporter mean when she

said,“The world is at Dylan’s fingertips”?

enjoys lots of successes 7 18

has lots of potential 19 45

Total score: 7–8 2 13

5–6 20 46

3-4 43 35

0–2 35 6

Commentary:

Year 8 students performed substantially better than year

4 students, especially on the more subtle or inferential

components of the task.


Chapter 3: Listening 21

Line Up

Approach: Station Level: Year 4 and year 8

Focus: Following instructions, logical thinking.

Resources: Computer programme on laptop computer.

Question/instructions:

This activity uses the computer.

Click on the button that says Line Up to begin the

task. The computer will tell you what to do.

% responses

y4 y8

Audio script:

The five men live in the five houses. Each man lives in his own

house. The instructions will tell you which house they live in.

You will hear each instruction once only. Follow the instructions

as you hear them, dragging the men and placing them in their

house.

[First Ph a s e]

◗ The man with the glasses is in the middle house.

◗ The man with the yellow hat is in house number 5.

◗ The man with white hair is NOT next to the man with the

glasses.

◗ The painter is beside the man with the glasses.

◗ The man with the black hat is between the man with the

glasses and the man with the yellow hat.

When you are ready, click the Next button.

One day the men all decided to change houses. Listen to the new

instructions. Drag each man into his new house.

[Second Ph a s e]

◗ The man with the yellow hat is in the second house from

your left.

◗ The painter is not next to the man with the yellow hat.

◗ The man with the glasses doesn’t live in a house with an

odd number.

◗ The man with the black hat is standing alongside only one

person.

◗ Put the last man in the empty house.

When you are ready, click the Finish button.

First phase: 5 correct 73 83

4 correct 10 9

3 correct 8 3

2 correct 5 1

1 correct 3 1

0 correct 1 0

Second phase: 4 or 5 correct 33 62

3 correct 14 8

2 correct 21 14

1 correct 26 13

0 correct 6 3

Total score: 9–10 31 61

7–8 27 19

5–6 31 17

3–4 8 3

0–2 3 0

Commentary:

The second phase, especially, was done much better by year 8 than year 4 students. The task involved significant

logical thinking, with some components involving guessing and subsequent correction if wrong.


22 NEMP Report 25: Listening and Viewing 2002

Zippos

Approach: Independent Level: Year 4 and year 8

Focus: Recalling information, drawing appropriate conclusions, representing information visually.

Resources: Audio recording on laptop computer.

Questions/instructions:

[This activity uses the computer for a soundtrack only.

There are no visuals provided on the computer.]

This activity uses the computer.

Click the Zippos button.

In this activity you are going to listen to some facts

about Zippos. Zippos are strange creatures that live

deep in caves. We don’t know what Zippos look like

but we do have some information about them.

Listen carefully to the information you hear about

Zippos and draw what you think they look like. Draw

your picture while you are hearing the information.

The video will give the information twice. You can

add to or change your drawing as you listen to the

information a second time.

I’ll play the video now.

Click the Play button.

Audio script:

In this activity you are going to draw a Zippo. Zippos

live in caves and never come into the light. Nobody has

ever seen a Zippo so we don’t know exactly what they

look like. You are going to hear some information

about Zippos. Use this information to draw what you

think a Zippo would look like.

You can start drawing as the information is given to

you. The information will be given twice. You can

check your drawing when the information is given the

second time.

Facts about Zippos:

◗ Zippos are round in shape. This helps them to roll

from place to place.

◗ They have 2 large round eyes that help them to

see in the dark caves. Each eye is made up of

four circles.

◗ Zippos use their large wings to fly about in the

cave, where they feed on spiders and insects.

◗ Zippos are friendly. They have big happy smiles.

◗ Zippos have 3 short hairy legs, and claws on their

feet. Now listen to the information again.

Features of drawing:

% responses

y4 y8

round in shape 72 84

two round eyes 84 88

3 or 4 circles inside eyes 57 83

wings 89 94

wings are large 61 75

smile/happy 86 93

3 legs 58 84

legs are short 70 84

legs are hairy 60 66

legs have claws (on their feet) 84 91

Total score: 9–10 28 58

7–8 42 31

5–6 21 8

0–4 9 3

Commentary:

Both year 4 and year 8 students managed well on this

task. Year 8 students did notably better on the finer

details, so that 30 percent more year 8 than year 4

students got a total score of 9 or 10.


Chapter 3: Listening 23

YEAR 4 EXEMPLARS – HIGH

YEAR 8 EXEMPLARS – HIGH


24 NEMP Report 25: Listening and Viewing 2002

Lexis

Approach: Station Level: Year 8

Focus: Understanding and following instructions.

Resources: Computer game on laptop computer.

Questions/instructions:

This activity uses the computer.

Click on the button that says Lexis to

begin the task. The computer will tell

you what to do.

[Each screen of the game requires the student to

perform one or more actions before being able to

progress to the next stage. There are a variety of

actions with varying degrees of difficulty e.g. locate

the correct object and move it to a specified location;

create a path to a destination.]

% responses

y8

Audio instructions:

Lexis and Zep have lost their dog, Wag. He has

wandered off near the dreaded castle. It is your

job to help Lexis and Zep by listening to the

instructions and finding your way through the

many rooms of the dreaded castle.

The Dreaded Castle: Put the red rock on the

highest gargoyle head to open the drawbridge.

[Tasks 1–2]

Task: 1 (find red rock) 97

2 (place on correct gargoyle) 90

3 (find correct path to red diamond) 23

4 (find gold ring) 76

5 (place on small rhino horn) 91

6 (press lowest blue button) 97

7 (press top blue button) 95

8 (correct opening of door) 99

9 (remove first tooth) 98

10 (remove second tooth) 96

11 (find ‘S’ cracked paving stone) 94

12 (find brick marked ‘X’) 77

13 (key in bottom lock) 76

14 (key in top lock) 80

The Main Hall: Make your way carefully,

avoiding the cracked tiles, and walk to the red

diamond to open the next door. Click on each

tile to show your path.

[Task 3]

The Dining Room: Using the gold ring, put

it on the smallest horn of the rhino’s head to

open the secret passage.

[Tasks 4–5]

Laboratory: Press the lowest blue button on

the panel to put the monster to sleep and then

on the top blue button to open the door.

[Tasks 6–7]

Dungeon: Using the blue glove, place it on

the door to open it but beware the monster.

[Task 8]

Total score: 13–14 42

11–12 40

9–10 15

less than 9 3

Commentary:

Apart from one component which was particularly

challenging, students enjoyed high levels of success

on this task. Only 18 percent of students got more

than 3 of the 14 components incorrect.

Skull Room: Carefully remove the two top

teeth of the skull’s mouth to open the secret

passage to the garden.

[Tasks 9–10]

The Garden Gate: You’ve found Wag but he’s

behind a gate. To open it, look under the ‘S’

cracked paving stone for the key to the bottom lock

and behind the lowest brick in the wall with an ‘X’

on it. Use these to open the gate and rescue Wag.

Well done. You’ve opened the gate and you’ve

rescued Wag. [Tasks 11–14]


Chapter 3: Listening 25

LINK TASK 1

Approach: One to one

Level: Year 4 and year 8

Focus: Evaluating viewpoints and opinions.

% responses

y4 y8

LINK TASK 2

Approach: One to one

Level: Year 4 and year 8

Focus: Recalling and retelling messages.

Link tasks 1–8

% responses

y4 y8

Total score: 10–14 3 9

7–9 30 42

4–6 48 45

0–3 19 4

Total score: 13–19 5 12

10–12 16 30

7–9 35 34

4–6 33 19

0–3 11 5

LINK TASK 3

Approach: One to one

Level: Year 4 and year 8

Focus: Recalling and summarising orally.

% responses

y4 y8

LINK TASK 4

Approach: One to one

Level: Year 4 and year 8

Focus: Interpretation and inference.

% responses

y4 y8

Total score: 12–16 4 14

9–11 20 32

6–8 37 31

3–5 23 18

0–2 16 5

Total score: 5–7 3 9

4 10 16

3 18 22

2 25 27

1 25 19

0 19 7

LINK TASK 5

Approach: Station

Level: Year 4 and year 8

Focus: Understanding and following instructions.

% responses

y4 y8

LINK TASK 6

Approach: Station

Level: Year 4 and year 8

Focus: Listening for specific information.

% responses

y8

Total score: 13–14 23 61

11–12 36 24

9–10 20 9

7–8 9 3

0–6 12 3

Total score: 8 10 51

7 20 26

5–6 35 18

0–4 35 5

LINK TASK 7

Approach: One to one

Level: Year 8

Focus: Recalling and sequencing instructions.

% responses

y4 y8

LINK TASK 8

Approach: Station

Level: Year 8

Focus: Distinguishing fact and opinion.

% responses

y8

Total score: 9–10 40

7–8 43

5–6 14

0–4 3

Total score: 7 22

6 28

4–5 29

2–3 17

0–1 4


26 Chapter 4

VIEWING

The assessments included nineteen tasks which asked the

students to view visual resources (some of which were

accompanied by words or a soundtrack) and to demonstrate

understanding of the messages conveyed, their purposes, the

contexts in which they were appropriate, or the particular

techniques used. Visual material is a prominent part of life

in our world. It takes many forms, such as illustrations in

books, photographs, comics and cartoons, posters, brochures,

advertisements, films and television programmes. Students

need to learn to make sense of this material, and to become

discriminating users of it.

Fourteen tasks were identical for both year 4 and year 8 students,

and one had some overlapping components for year 4 and 8

students but additional components for year 8 students only. One

task was administered only to year 4 students, and three only to

year 8 students. Six are trend tasks (fully described with data for

both 1998 and 2002), five are released tasks (fully described with

data for 2002 only), and eight are link tasks (to be used again in

2006, so only partially described here).

The tasks are presented in the three sections: trend tasks, then

released tasks and finally link tasks. Within each section, tasks

administered to both year 4 and year 8 students are presented

first, followed by tasks administered only to year 4 students and

then tasks administered only to year 8 students.

Averaged across 129 task components administered to both year

4 and year 8 students, 9 percent more year 8 than year 4 students

succeeded with these components. Year 8 students performed

better on 110 of the 129 components. The components with the

largest differences generally involved judgement or inference,

rather than observation and reporting.

The trend analyses showed small gains since 1998. Averaged

across 21 task components attempted by year 4 students in both

years, 2 percent more students succeeded in 2002 than in 1998,

with gains on 12 components and losses on 7 components.

These gains are not large enough to be regarded as significant.

At year 8 level, with 29 task components included in the analysis,

3 percent more students succeeded in 2002 than in 1998, with

gains on 21 components and losses on 8 components. Overall,

this suggests a modest gain for year 8 students over the four-year

period.

Both year 4 and year 8 students often achieved quite high

performance levels on task components that involved observing,

recalling and using specific factual information. Predictably,

they were less successful where the task components involved

interpretation or evaluation of visual messages, or of the

intentions of the designers of those messages. These latter

components usually were handled substantially better by year

8 than year 4 students.


Chapter 4: Viewing 27

trend task

It’s Cool to Read

Approach: One to one Level: Year 4 and year 8

Focus: Identifying persuasive images.

Resources: 9 pictures.

Questions/instructions:

Give student the set of 9 pictures

3

4

1

2

7

5

6

9

8

Imagine that you are making a poster to try to get

young children more interested in reading books

for fun.

1. From this collection of 9 photographs I want you

to choose two that you think would be specially

good for your poster. Which two pictures would

you choose?

Allow time

2. Now I would like you to explain

to me why you chose these two

pictures for your poster.

Justification, first picture:

% responses

2002 (’98) 2002 (’98)

year 4 year 8

strong 11 (7) 27 (20)

moderate 73 (73) 67 (72)

weak 16 (20) 6 (8)

Look at the other pictures and decide

on one that you wouldn’t use for your

poster.

3. Which one would you choose?

4. Explain to me why you wouldn’t

use that picture?

% responses

2002 (’98) 2002 (’98)

year 4 year 8

Justification: strong 8 (5) 19 (11)

moderate 73 (70) 71 (73)

weak 19 (25) 10 (16)

Total score: 5–6 8 (5) 20 (11)

4 10 (6) 17 (15)

Justification, second picture:

strong 11 (9) 24 (16)

moderate 72 (69) 68 (74)

weak 17 (22) 8 (10)

3 51 (45) 49 (51)

2 17 (25) 9 (17)

0–1 14 (19) 5 (6)

Commentary:

The pictures nominated most for Question 1 were pictures 1, 4 and 9. The pictures nominated as least suitable

(Question 3) were pictures 8 (at both year levels) and 6 (year 8 only). About 15 percent more year 8 than year 4

students gave justifications for their choices that were judged to be ‘strong’ justifications.


28 NEMP Report 25: Listening and Viewing 2002

The Wolf

trend task

Approach: One to one Level: Year 4 and year 8

Focus: Identifying visual and auditory effects used to create mood.

Resources: Video recording on laptop computer.

Questions/instructions:

This activity uses the computer.

Click the The Wolf button.

I’m going to show you a clip from a cartoon called Peter and the Wolf, then we’ll talk about it.

Click the Play button.

The cartoon shows that the wolf is scary and really

bad. I want you to think of the special techniques or

effects that were used to make the wolf seem scary

and bad.

I’ll write down all of the ideas you can think of, so that

you can check what you’ve said.

Features mentioned:

% responses

2002 (’98) 2002 (’98)

year 4 year 8

bad weather, swirling wind

and snow, dark 27 (13) 36 (33)

footprints 16 (18) 17 (25)

dark shadowy atmosphere,

trees/forest 34 (12) 41 (27)

something moving behind trees 20 (19) 35 (33)

close up of wolf’s face

(with snarling noise, big teeth, yellow eyes) 86 (88) 88 (92)

spooky music 39 (37) 67 (60)

How does the video make the wolf seem scary and bad?

As the student says each idea, write it down and read

aloud.

Where 2 or more ideas are given as one, suggest that they

be written separately.

Overall quality of response:

% responses

2002 (’98) 2002 (’98)

year 4 year 8

excellent 0 (0) 1 (0)

good 13 (7) 23 (16)

moderate 59 (58) 59 (64)

poor 28 (35) 17 (20)

Total score: 8–15 7 (2) 17 (13)

6–7 13 (7) 22 (15)

4–5 38 (43) 34 (44)

2–3 27 (28) 21 (22)

0–1 15 (20) 6 (6)

Commentary:

Both at year 4 and at year 8 level, there was a small overall increase in performance between 1998 and 2002. Students

appeared not very aware of the more subtle techniques used in the scene to create a scary mood but, in part, this

may have arisen from the focus of the question on the wolf.


Chapter 4: Viewing 29

trend task

Santa Gets Ready

Approach: One to one Level: Year 4 and year 8

Focus: Ordering pictures to tell a story.

Resources: 8 picture cards.

Questions/instructions:

Give student the set of pictures in order 1–8

Look carefully at all 8 pictures. They tell a story about

Father Christmas getting ready, but they are not in the

right order.

1

2

1. Put the pictures into the right

order. Don’t take any notice of the

numbers on the cards.

Allow time for the student to arrange

pictures.

Record the card numbers before asking

the next question.

% responses

2002 (’98) 2002 (’98)

year 4 year 8

Correct order: (4, 8, 2, 7, 1, 5, 3, 6) 51 (46) 65 (69)

3

2. How do you know these are now in

the right order?

prompt: What are the clues?

not marked • •

If you want to make any changes you

can do that now.

4

5

Point to card numbered 3.

3. How do you know this one goes

here?

If the student has made any changes,

record the changes below the existing

answers.

6

4 clues: (all clothes on; bag strap visible;

still inside house; saying goodbye)

mentioned all 4 clues 2 (1) 1 (3)

mentioned 3 clues 8 (6) 12 (15)

mentioned 2 clues 29 (31) 35 (36)

mentioned 1 clue 49 (45) 44 (36)

7

any other response 12 (17) 8 (10)

8

Commentary:

While half or more of the students at both levels could

place the 8 pictures into the correct order, only a small

percentage could thoroughly justify the placement

of the second to last picture in the sequence. Year

8 students performed only a little better than year 4

students in this regard.


30 NEMP Report 25: Listening and Viewing 2002

TV Commercials

trend task

Approach: One to one Level: Year 4 and year 8

Focus: Understanding persuasive techniques.

Resources: 2 television advertisements on laptop computer.

Questions/instructions:

This activity uses the computer.

Click the TV Commercials button.

I’m going to show you two television

commercials.

Click the Play Button, pause the

video when ‘Pause’ sign appears.

1. What sort of feelings do you think

this commercial is trying to give?

prompt: How do they want people

to feel when they watch this

commercial?

% responses

2002 (’98) 2002 (’98)

year 4 year 8

scary/spooky feelings • 84 (67)

The commercial is trying to give scary

or spooky feelings, so that people

might feel afraid.

2. What techniques have been used in

the commercial to cause those sorts

of feelings?

black and white only • 31 (24)

scary music • 44 (37)

spooky appearance of characters • 65 (79)

bizarre behaviour of characters • 53 (43)

Overall quality of response:

3. Why might an advertiser use a

scary commercial to try to sell

something?

set at night • 46 (34)

good • 18 (10)

moderate • 62 (56)

poor • 20 (34)

capture attention, make memorable • 40

Now let’s watch another commercial.

Click the Play Button.

4. Why has Telecom made this

commercial?

% responses

2002 (’98) 2002 (’98)

year 4 year 8

to encourage calls to friends in

other countries 14 (18) 29 (34)

Telecom made this commercial to

encourage people to use their phones

to call friends in other countries.

5. Apart from the message at the end

of the commercial, what has the

rest of the commercial got to do

with ringing up people in other

countries?

makes you think of other countries

(e.g. animals, music) • 31 (37)

just designed to catch attention • 12 (9)

6. Why do you think Telecom made

the commercial like this — with

lots of cute animals?

Possible points: – capture attention

– keep attention

– cause amusement

2 or more relevant ideas 4 (6) 17 (12)

1 relevant idea 28 (23) 49 (48)

no relevant ideas 68 (71) 34 (40)

Total score: 10–14 • 11 (10)

8–9 • 21 (14)

6–7 • 33 (32)

4–5 • 24 (25)

0–3 • 11 (19)

• not asked for year 4

Commentary:

Year 4 students were only shown the second commercial and were only asked questions 4 and 6. It is notable that

the students did not seem particularly aware of the underlying purpose of the Telecom advertisement.


Chapter 4: Viewing 31

[First commercial – Year 8 only]

Video script:

Woman: This is most strange... oh, look.

[Old man at side of road]

Man: (to old man) I think we’re lost.

[Evil laughter from old man]

Man & Woman: How odd!

[Couple drive away. Old lady at side of road further on]

Man:

The lady with the lamp. I’m sure she’ll help us.

[Insane gibberish from old lady]

Man: (nervously) Oh, oh... thank you... we’ve found it!

[Couple drive away. Three strange men at side of road further on]

Man:

Ah, I’m sure these people will help

Excuse us...(to men).

[Men stare blankly into car]

Woman: Ahhhh! (strangled cry)

[Second commercial – Year 4 and year 8]

Video script:

[Irish dance music only throughout most of video]

Line up friends and family in the U.K. and Europe this weekend for

a huge chat, with Telecom.

Call anyone, anywhere in the U.K. and Europe from 6pm Friday to

midnight Sunday and only pay for the first 20 minutes. The rest of

the call is free.

Excludes 059 calls.


32 NEMP Report 25: Listening and Viewing 2002

Bedroom Plan

trend task

Approach: Station Level: Year 4

Focus: Interpreting a plan.

Resources: Laminated plan.

Questions/instructions:

Here is a plan of Nick’s bedroom. Look at the plan, then put rings around the best answers.

1. What is in the middle of the room?

a lamp

a rug

a book case

a fish tank

2. The head of Nick’s bed is ...

on the North wall

on the East wall

3. The chest of drawers is ...

near the wardrobe

by the toy box

on the South wall

% responses

2002 (’98)

year 4

a rug 94 (94)

on the East wall 74 (75)

on the North wall

on the North wall 87 (87)

4. How many windows are there in

the room?

1 2 3 4 2 65 (67)

5. What would you put in the thing

shown on the East wall?

books

toys

fish

rubbish

toys 74 (76)

6. What would you find between the

table and the bookcase?

toy box

rug

chair

rubbish bin

% responses

2002 (’98)

year 4

rubbish bin 91 (93)

7. What would you see first when you

came through the door?

table

toy box

fish tank

book case

toy box or bookcase (or both) 87 (85)

Total score: 7 35 (35)

6 32 (32)

5 17 (18)

3–4 12 (13)

0–2 4 (2)

Commentary:

The 1998 and 2002 results are almost identical.


Chapter 4: Viewing 33

trend task

House Plan

Approach: Station Level: Year 8

Focus: Interpreting a plan.

Resources: Pictures in answer booklet.

Questions/instructions:

Look at the picture and plan of the house then answer

the questions.

1. Put a cross on the plan to

show the front door.

% responses

2002 (’98)

year 8

correct 76 (79)

2. How many bedrooms are

there? 3 99 (96)

3. Which room has the biggest

wardrobe?

bedroom 1, bed 1

(4 x 3.5, rm 1) 92 (90)

N

4. Draw a line on the plan to

show the shortest way from

the laundry to the lounge.

correct 73 (69)

W

S

E

5. What direction are you facing

when you are in the kitchen

looking out to the dining area?

North 84 (82)

6. There is a clothes line beside

the house. Draw the clothes

line in the right place on the

plan.

drawn in appropriate region 53 (49)

7. The person who drew the plan

has not labelled the part shown

like this. What is it?

sundeck, patio, tiled area, etc


[Not marked. Many students noted

the materials used rather than the

function of the area]

Commentary:

Year 8 students performed similarly on this task in 1998 and 2002.


34 NEMP Report 25: Listening and Viewing 2002

Māori Gods

Approach: One to one Level: Year 4 and year 8

Focus: Interpreting symbolic elements.

Resources: Prompt card, 3 pictures.

Māori Gods Prompt Card

Questions/instructions:

Show prompt card.

This is Tangaroa, the Māori god of the sea. You can tell this is

the god of the sea because he has a fish tail, hair like seaweed

and is blue and green like the sea.

This is a Māori god of the sea.

The picture shows he is the god

of the sea because of:

the fish tail,

the seaweed hair,

and the blue and green colours

like the sea.

1

Show picture 1.

1. What do you think

he is the god of?

[Tawhiri-matea

– god of wind/storms/

weather/clouds/rain]

% responses

y4 y8

name and ‘god of’

both correct 0 1

‘god of’ correct but

not name 42 58

3

% responses

y4 y8

2. How does the picture show that?

[judged in relation to student’s answer to 1]

2

appropriately argued, using

multiple features 46 66

appropriately argued,

using one feature 43 30

Show picture 2.

3. What do you think

he is the god of?

[Tumatauenga

– god of man/people or

war/fighting]

4. How does the picture show that?

name and ‘god of’

both correct 0 0

‘god of’ corrrect

but not name 19 28

[judged in relation to student’s answer to 3]

appropriately argued, using

multiple features 35 47

appropriately argued,

using one feature 46 39

Show picture 3.

5. What do you think he is the god of?

[Rongomatane/Rongomaraero

– god of peace/happiness or god of

gardening/farming/agriculture, kumara/arts]

name and ‘god of’ both correct 0 0

‘god of’ corrrect but not name 3 14

6. How does the picture show that?

[judged in relation to student’s answer to 5]

appropriately argued,

using multiple features 34 54

appropriately argued,

using one feature 52 36

Total score: 8–12 5 13

6–7 24 40

4–5 36 27

2–3 28 17

0–1 7 3

Commentary:

Students at both year levels showed substantial ability

to reason using visual cues but little prior knowledge of

these images.


Chapter 4: Viewing 35

Sweet Stall

Approach: One to one Level: Year 4 and year 8

Focus: Evaluation of a visual resource.

Resources: Video recording on laptop computer.

Questions/instructions:

This activity uses the computer.

Click the Sweet Stall button.

Imagine some children made some sweets. They want to sell

them to other children in the school. They have made a video

to advertise their sweets. This is their first go at making the

video.

Let’s watch the video now. Click the Play button.

Video script:

These sweets are for sale in Room 3. They cost $2 a

bag. They will be sold at lunchtime on Thursday

and Friday.

You can buy one bag for $2 or 3 bags for $5.

Be quick before they are all sold out!

Come to Room 3 for the best sweets in school!

[Images were of deliberately poor quality]

1. What was good about their video?

% responses

y4 y8

clear voices 20 40

well rehearsed/presented 20 31

variety of views (zooming) 0 1

Overall rating for comments on strength:

The children say that this video is not good

enough.

2. What needs to be improved?

Allow time.

strong 0 1

moderate 8 18

weak 92 81

can’t see faces, too dark, camera

pointing at window 50 82

lots of distracting camera movement 11 38

camera on angle to vertical 5 15

camera not focused 4 19

children not centred in pictures 2 12

children looking to side, not towards camera 2 15

Overall rating for comments on strength:

% responses

y4 y8

strong 0 8

moderate 11 38

Now I will play the video a second time.

Watch it again and see if there is anything

else that you can tell the children on how to

improve their video.

Click the Play button.

Allow time.

weak 89 56

Total score: 6–13 1 17

4–5 10 26

2–3 24 27

1 31 23

0 34 7

Commentary:

Year 8 students performed much better than year 4

students, but both groups tended to focus on just the

most obvious features and not consider other aspects.


36 NEMP Report 25: Listening and Viewing 2002

Weet–Bix Card

Approach: One to one Level: Year 4 and year 8

Focus: Thinking critically about the intentions of visual messages (advertising).

Resources: Weet-Bix card.

Questions/instructions:

% responses

y4 y8

Place the Weet-Bix card in front of the student.

This card shows words and pictures which

are trying to give messages to people who

look at the Weet–Bix box.

I want you to think about the pictures rather

than the words.

1. Try to tell me the messages that the

pictures are trying to give. I will write

them down for you.

Record student’s responses.

Now I’ll read the messages you have found

about the Weet–Bix card. If you want to

change any of them, you can tell me.

Read things recorded to the student.

2. Are there any changes you would like to

make?

Make any changes offered by the student.

Messages identified:

Weet-Bix makes you... happy 9 23

healthy/strong 43 35

a good athlete 15 15

energetic/fit 37 53

a winner 30 37

you should eat lots of Weet-Bix 9 10

Weet-Bix is good for you/your heart 17 14

Weet-Bix sponsor tryathlon 13 17

3. Do you think the messages are true?

Why do you say that?

not marked • •

% responses

y4 y8

Total score: 4–8 5 10

3 21 21

2 30 38

1 30 24

0 14 7

4. Why do you think the makers of Weet–Bix

want to give these messages to people?

so you buy more Weet-Bix 43 63

Commentary:

Students tended to settle for commenting on just one or

two features, rather than looking and commenting more

thoroughly. There was little difference between year 4

and year 8 students apart from greater awareness of the

advertiser’s purpose (product sales).


Chapter 4: Viewing 37

Poster

Approach: Station Level: Year 4 and year 8

Focus: Thinking critically about the intentions of visual messages.

Resources: Poster, 3 stickers.

Questions/instructions:

Look carefully at the poster. It gives

important messages.

What messages is the poster trying

to give?

Try to think of 3 key messages it is

telling you.

Write each message on a sticker.

When you have finished stick the stickers

on the black and white poster.

% responses

y4 y8

Key messages relating to balance:

important to balance diet/foods we eat 10 40

important to balance food and exercise 1 4

Other messages:

food and exercise affect our heart 1 2

exercise matters 36 58

food matters 50 69

exercise is fun 1 1

food is fun 0 0

exercise gives energy 0 0

food gives energy 0 2

Total score: 4–11 4 15

3 4 22

2 29 38

1 23 16

0 40 9

Commentary:

Most year 4 students and many year 8 students missed

the focus on balance entirely. Those who did focus on

balance tended to mention only balancing foods eaten,

not balancing food and exercise.

STUDENT EXAMPLES

Do lots of

exercise that will

keep you not

getting fat.

It

is telling you

to have a

balanced diet buy

eating things like

milk, watermelon,

cheese and

meat.

Exercise and

keep fit. Go out for

a run once

in a while.

But have FUN.

Eat right foods

like fruit &

vegetables & meat.

Keep healthy but

still have fun

in your life.


38 NEMP Report 25: Listening and Viewing 2002

Minties Moments

Approach: One to one Level: Year 8

Focus: Thinking critically about the intentions and effects of advertising.

Resources: Video recording on laptop computer.

[Video shows amusing mishaps in a variety of sporting situations; soundtrack is

music only throughout until last scene when Minities jingle plays.]

Questions/instructions:

This activity uses the computer.

Click the Minties Moments button.

We will begin this activity by watching a video of a Minties

advert. As you watch the advert, think carefully about what it

is trying to tell you. When you have watched the ad, I will ask

you some questions about it.

Click the Play button.

% responses

y8

1. What are they telling you in this ad?

2. Do you agree with what the ad is telling you?

3. Why do you say that?

Markers considered all three responses.

[Message seems to be that Minties make you feel better when something

bad, shocking, embarrassing has happened.]

How well did student show they understood this message?

well 20

moderately well 28

poorly 52

student mentioned advertiser’s goal to get

people to buy Minties 21

Commentary

No commentary.


Chapter 4: Viewing 39

LINK TASK 9

Approach: One to one

Level: Year 4 and year 8

Focus: Thinking critically about visual messages.

% responses

y4 y8

LINK TASK 10

Approach: One to one

Level: Year 4 and year 8

Focus: Evaluating visual design features.

Link tasks 9–16

% responses

y4 y8

Total score: 12–24 2 13

9–11 8 24

6–8 23 34

3–5 43 23

0–2 24 6

Total score: 6–14 5 9

4–5 17 36

2–3 54 44

0–1 24 11

Approach: One to one

Level: Year 4 and year 8

Focus: Retelling a scene.

LINK TASK 11

% responses

y4 y8

LINK TASK 12

Approach: One to one

Level: Year 4 and year 8

Focus: Analysing symbolic visual representations.

% responses

y4 y8

Total score: 12–15 2 5

9–11 35 32

6–8 50 54

3–5 13 9

0–2 0 0

Total score: 8–12 0 2

6–7 9 17

4–5 37 39

2–3 44 37

0–1 10 5

LINK TASK 13

Approach: One to one

Level: Year 4 and year 8

Focus: Obtaining and analysing visual information.

% responses

y4 y8

LINK TASK 14

Approach: One to one

Level: Year 4 and year 8

Focus: Thinking critically about advertising.

% responses

y4 y8

Total score: 8–9 6 11

6–7 31 51

4–5 49 32

2–3 13 6

0–1 1 0

Total score: 4–5 2 19

3 10 28

2 41 28

1 36 20

0 11 5

LINK TASK 15

Approach: Station

Level: Year 4 and year 8

Focus: Reading body language.

% responses

y4 y8

LINK TASK 16

Approach: One to one

Level: Year 8

Focus: Interpreting and comparing advertisements.

% responses

y8

Total score: 4 34 59

3 25 25

2 26 12

1 9 3

0 6 1

Total score: 12–19 5

9–11 25

6–8 43

3–5 21

0–2 6


40 Chapter 5

PERFORMANCE OF SUBGROUPS

Although national monitoring has been designed

primarily to present an overall national picture of

student achievement, there is some provision for

reporting on performance differences among subgroups

of the sample. Seven demographic variables are available

for creating subgroups, with students divided into two

or three subgroups on each variable, as detailed in

Chapter 1 (p5).

The analyses of the relative performance of subgroups

used an overall score for each task, created by adding

together scores for appropriate components of the

task.

Where only two subgroups were compared, differences

in task performance between the two subgroups were

checked for statistical significance using t-tests. Where

three subgroups were compared, one way analysis of

variance was used to check for statistically significant

differences among the three subgroups.

Because the number of students included in each analysis

was quite large (approximately 450), the statistical

tests were quite sensitive to small differences. To reduce

the likelihood of attention being drawn to unimportant

differences, the critical level for statistical significance

was set at p = .01 (so that differences this large or larger

among the subgroups would not be expected by chance

in more than one percent of cases).

For the first three of the seven demographic variables,

statistically significant differences among the subgroups

were found for less than 12 percent of the tasks at both

year 4 and year 8. For the remaining four variables, statistically

significant differences were found on more than

29 percent of tasks at one or both levels. In the detailed

report below, all “differences” mentioned are statistically

significant (to save space, the words “statistically

significant” are omitted).

School Size

Results were compared from students in large, medium

sized, and small schools (exact definitions were given

in Chapter 1). For year 4 students, there were no differences

among the subgroups on any of the 14 listening

tasks, and a difference on only 1 of the 16 viewing tasks:

students from main centres scored lowest on It’s Cool

to Read (p27). For year 8 students, there were no differences

on any of the 17 listening tasks or 18 viewing

tasks.

School Type

Results were compared for year 8 students attending

full primary and intermediate schools. There were no

differences between these two subgroups on any of the

18 viewing tasks, with a difference on just 1 of the 17

listening tasks. Students from full primary schools scored

higher than students from intermediate schools on

The Wind and the Sun (p18).

Community Size

Results were compared for students living in

communities containing over 100,000 people (main

centres), communities containing 10,000 to 100,000

people (provincial towns), and communities containing

less than 10,000 people (rural areas).

For year 4 students, there were no differences

among the three subgroups on any of the 16 viewing

tasks, with a difference on just 1 of the 14 listening

tasks. Students from main centres scored lowest on

Link Task 1 (p28).

For year 8 students, there were no differences among

the three subgroups on any of the 18 viewing tasks, with

differences among them on just 2 of the 17 listening

tasks. Students from rural areas scored lowest and

students from provincial towns highest on Line Up

(p21), while students from main centres scored lowest

and students from provincial towns highest on Link

Task 7 (p25).

Zone

Results achieved by students from Auckland, the rest of

the North Island, and the South Island were compared.

For year 4 students, there were differences among the

three subgroups on 5 of the 14 listening tasks: Cats Sleep

Anywhere (p16), Link Task 1 (p25), Link Task 3 (p25),

Link Task 4 (p25), and Link Task 5 (p25). Students

from the South Island scored highest on 4 of these tasks,

with students from Auckland lowest on 4 of them. There

were also differences on 7 of the 16 viewing tasks: The

Wolf (p28), Santa Gets Ready (p29), TV Commercials

(p30), Māori Gods (p34), Sweet Stall (p35), Link Task 9

(p39), and Link Task 10 (p39). Students from the South

Island scored highest on all 7 of these viewing tasks, with

students from Auckland lowest on 5 of them.

For year 8 students, there was a difference among the

three subgroups on just 1 of the 17 listening tasks:

students from Auckland scored lowest and students

from the rest of the North Island highest on Zippos

(p22). Similarly, there was a difference on just 1 of the

18 viewing tasks: students from Auckland scored lowest

and students from the South Island highest on Link Task

15 (p39).

Gender

Results achieved by male and female students were

compared.

For year 4 students, there were differences between boys

and girls on 2 of the 14 listening tasks. Girls scored higher

than boys on Link Task 3 (p25) and Link Task 4 (p25).

There was also a difference on 1 of the 16 viewing tasks,

with girls scoring higher than boys on Poster (p37).

For year 8 students, there were differences between boys

and girls on 5 of the 17 listening tasks. Girls scored higher


Chapter 5: Performance of Subgroups 41

than boys on Phone Message (p14), Zippos (p22), Link

Task 2 (p25), Link Task 3 (p25), and Link Task 4 (p25).

There were also differences on 2 of the 18 viewing tasks,

with girls scoring higher than boys on Weet-Bix Card

(p36) and Poster (p37).

Student Ethnicity

Results achieved by Māori and non-Māori students were

compared.

For year 4 students, there were differences on 5 of

the 14 listening tasks. Non-Māori students scored

higher than Māori students on The Wind and the Sun

(p18), Drummer Dylan (p20), Zippos (p22), Link

Task 2 (p25), and Link Task 5 (p25). There were also

differences on 6 of the 16 viewing tasks. Non-Māori

students scored higher than Māori students on It’s Cool

to Read (p27), Santa Gets Ready (p29), Bedroom Plan

(p32), Link Task 9 (p39), Link Task 10 (p39), and Link

Task 14 (p39).

For year 8 students, there were differences on 3 of

the 17 listening tasks. Non-Māori students scored

higher than Māori students on Line Up (p21), Link

Task 2 (p25), and Link Task 6 (p25). There were also

differences on 6 of the 18 viewing tasks. Non-Māori

students scored higher than Māori students on House

Plan (p33), Sweet Stall (p35), Poster (p37), Link Task 9

(p39), Link Task 12 (p39), and Link Task 14 (p39).

Socio-Economic Index

Schools are categorised by the Ministry of Education

based on census data for the census mesh blocks where

children attending the schools live. The SES index takes

into account household income levels, categories of

employment, and the ethnic mix in the census mesh

blocks. The SES index uses ten subdivisions, each

containing ten percent of schools (deciles 1 to 10). For

our purposes, the bottom three deciles (1-3) formed

the low SES group, the middle four deciles (4-7) formed

the medium SES group, and the top three deciles (8-10)

formed the high SES group. Results were compared for

students attending schools in each of these three SES

groups.

For year 4 students, there were differences among the

three subgroups on 10 of the 14 listening tasks and 8 of

the 16 viewing tasks. Because of the large number of

tasks involved, they will not be listed here. In all cases,

students in the low SES schools performed worst. While

students from high SES schools generally did better than

students from medium SES school, these differences

were usually smaller than the differences between

students from low and medium SES schools.

For year 8 students, there were differences among the

three subgroups on 10 of the 17 listening tasks and 11

of the 18 viewing tasks. For about half of these tasks,

the prominent feature was the low performances of

students in the low SES schools, with only modest

differences between students from medium and high SES

schools. For the remaining tasks showing differences, the

performance gaps were more evenly distributed or larger

between students from medium and high SES schools.

Summary

School size, school type (full primary or intermediate)

and community size did not seem to be important factors

predicting achievement on listening and viewing tasks.

South Island students performed better than Auckland

students on about 40 percent of the listening and viewing

tasks at year 4 level, but only about 5 percent of the

year 8 tasks. At both year levels, girls performed better

than boys on some tasks, with the proportion of these

tasks increasing somewhat from year 4 to year 8 (14 to 29

percent for listening, 6 to 11 percent for viewing). Non-

Māori students outperformed Māori students on about 35

percent of the viewing tasks at both year levels and on

the listening tasks at year 4 level, but this dropped to 18

percent of the year 8 listening tasks. The SES index based

on school deciles showed the strongest pattern of differences,

with differences on 50 to 70 percent of listening

and viewing tasks at both year levels.

Between 1998 and 2002, there have been noteworthy

changes in subgroup differences for four of the seven

variables. The only variable showing increased disparity

was geographic zone, and that only at year 4 level,

with the performance gap between South Island and

Auckland students increasing between 1998 and 2002

(from 13% to 36% of listening tasks, and from 22% to

44% of viewing tasks). On the other hand, there were

substantial reductions in subgroup differences for three

variables: gender, ethnicity and the SES index based on

school deciles. Over the four-year period, the percentage

of viewing tasks on which girls performed better than

boys decreased from 22 percent to 6 percent for year

4 students and from 29 percent to 11 percent for year

8 students. The percentage of tasks on which Māori

students scored lower than other students decreased

substantially for listening and viewing tasks at both

year levels (50% to 36% for year 4 listening tasks, 33%

to 18% for year 8 listening tasks, 67% to 38% for year 4

viewing tasks, and 57% to 33% for year 8 viewing tasks).

Similarly, the percentage of tasks on which

students from low decile schools scored

significantly lower than students from

high decile schools decreased for

both sets of tasks at both year

levels (87% to 71% for year 4

listening tasks, 78% to 59%

for year 8 listening tasks,

100% to 50% for year

4 viewing tasks, and

86% to 61% for year 8

viewing tasks).


42 Chapter 6

PACIFIC SUBGROUPS

An additional feature in national monitoring since

1999 has been the commitment to look directly at

the achievement of Pacific students in New Zealand

primary and intermediate schools. These students

were among the samples in NEMP assessments between

1995 and 1998, but not in sufficient numbers to allow

their results to be reported separately. At the request

of the Ministry of Education, in each year since 1999

NEMP has selected special additional samples of 120

year 4 students and 120 year 8 students, so that the

achievement of Pacific students could be assessed

and reported. The augmented samples are too small,

however, to allow separate reporting on students from

different Pacific nations (such as Samoa, Tonga, and

Fiji).

All schools in the main

NEMP year 8 sample that

had 15 percent or more

Pacific students (as classified

in school records) were

selected. All other schools

nationally with at least 12

year 8 students and at least 15 percent Pacific students

in their total roll were identified, and an additional

random sample of 10 schools drawn from this list.

A similar procedure was followed at year 4 level,

except that schools already chosen at year 8 level were

excluded from the sampling list. From each specially

sampled school, 12 students (in 3 groups of 4) were

sampled, confirmed and assessed using exactly the

same procedures as in the main sample. The students’

performances were also scored in the same manner as

the performances of students in the main sample.

The results for Pacific, Māori, and other students in the

schools with more than 15 percent Pacific students

were then compared. Because all of the schools chosen

for these analyses have at least 15 percent Pacific

students, the results only apply to students at schools

like these.

Differences among the three ethnic groups of students

were checked for statistical significance using one way

analysis of variance on the overall scores for each task.

Each analysis compared the performance of about

45 Pacific students, 25 Māori students and 35 other

students. The critical level for statistical significance

was set at p = .05 (so that differences this large or larger

among the subgroups would not be expected by chance

in more than five percent of cases). Where statistical

significance occurred, Tukey tests were used to identify

which groups differed significantly.

The mean scores for each group on each task are

presented in the tables following, together with the

standard deviations for all students in this sample.

Statistically significant differences are clearly

indicated.

YEAR 4

Average (mean) marks for year 4 students, attending

schools enrolling at least fifteen percent Pacific students,

who are classified as Pacific students, Māori students or

other students.

Statistically significant (p


Chapter 6: Pacific Subgroups 43

YEAR 8

Average (mean) marks for year 8 students, attending

schools enrolling at least fifteen percent Pacific students,

who are classified as Pacific students, Māori students or

other students.

Statistically significant (p


44 APPENDIX

THE SAMPLE OF SCHOOLS AND STUDENTS IN 2002

Main samples

In 2002, 2868 children from 249 schools were in the

main samples to participate in national monitoring. Half

were in year 4, the other half in year 8. At each level,

120 schools were selected randomly from national lists

of state, integrated and private schools teaching at that

level, with their probability of selection proportional

to the number of students enrolled in the level. The

process used ensured that each region was fairly

represented. Schools with fewer than four students

enrolled at the given level were excluded from these

main samples, as were special schools and Māori

immersion schools (such as Kura Kaupapa Māori).

Early in May 2002, the Ministry of Education provided

computer files containing lists of eligible schools with

year 4 and year 8 students, organised by region and

district, including year 4 and year 8 roll numbers drawn

from school statistical returns based on enrolments at

1 March 2002.

From these lists, we randomly selected 120 schools with

year 4 students and 120 schools with year 8 students.

Schools with four students in year 4 or 8 had about a

one percent chance of being selected, while some of

the largest intermediate (year 7 and 8) schools had a

more than 90 percent chance of inclusion. In the four

cases where the same school was chosen at both year 4

and year 8 level, a replacement year 4 school of similar

size was chosen from the same region and district, type

and size of school.

Additional samples

From 1999 onwards, national monitoring has included

additional samples of students to allow the performance

of special categories of students to be reported.

To allow results for Pacific students to be compared

with those of Māori students and ‘other’ students, 10

additional schools were selected at year 4 level and

10 at year 8 level. These were selected randomly from

schools that had not been selected in the main sample,

had at least 15 percent Pacific students attending the

school, and had at least 12 students at the relevant year

level.

To allow results for Māori students learning in Māori

immersion programmes to be compared with results

for Māori children learning in English, 10 additional

schools were selected at year 8 level only. They were

selected from Māori immersion schools (such as Kura

Kaupapa Māori) that had at least four year 8 students,

and from other schools that had at least four year 8

students in classes classified as Level 1 immersion (80

to 100 percent of instruction taking place in Māori).

Only students that the schools reported to be in at least

their fifth year of immersion education were included

in the sampling process.

Pairing small schools

At the year 8 level, 5 of the 120 chosen schools in the

main sample had less than 12 year 8 students. For each

of these schools, we identified the nearest small school

meeting our criteria to be paired with the first school.

Wherever possible, schools with 8 to 11 students were

paired with schools with 4 to 7 students, and vice versa.

However, the travelling distances between the schools

were also taken into account. Four of the 10 schools in

the year 8 Māori immersion sample also needed to be

paired with other schools of the same type.

Similar pairing procedures were followed at the year 4

level. Four pairs were required in the main sample of

120 schools.

Contacting schools

Late in May, we attempted to telephone the principals

or acting principals of all schools in the year 8 samples

(excluding the 14 schools in the Māori immersion

sample). We made contact with all schools within a

week.

In our telephone calls with the principals, we briefly

explained the purpose of national monitoring, the

safeguards for schools and students, and the practical

demands that participation would make on schools

and students. We informed the principals about the

materials which would be arriving at the school (a

copy of a 20-minute NEMP videotape plus copies for

all staff and trustees of the general NEMP brochure and

the information booklet for sample schools). We asked

the principals to consult with their staff and Board of

Trustees and confirm their participation by the end of

June.

A similar procedure was followed at the end of July

with the principals of the schools selected in the year 4

samples, and they were asked to respond to the invitation

by the end of August. The principals of the 14 schools

in the Māori immersion sample were contacted in early

August and asked to respond by early September. They

were sent brochures in both Māori and English.

Response from schools

Of the 283 schools originally invited to participate,

277 agreed. Three schools in the main year 8 sample

declined to participate: an intermediate school because

of severe space problems, a brand new independent

school because of the challenges associated with getting

established, and a full primary school because it was its

first year as a recapitated school and there had been a

major upheaval in their year 8 class. One school in the

year 8 Pacific sample was replaced because of major

building work and a bereavement that had disrupted the

work of the school. In the Māori immersion sample, two

schools chose not to participate and were replaced by

nearby schools.


Appendix 45

Sampling of students

With their confirmation of participation, each school

sent a list of the names of all year 4 or year 8 students

on their roll. Using computer generated random

numbers, we randomly selected the required number

of students (12, or 4 plus 8 in a pair of small schools),

at the same time clustering them into random groups

of four students. The schools were then sent a list

of their selected students and invited to inform us if

special care would be needed in assessing any of those

children (e.g. children with disabilities or limited skills

in English).

At the year 8 level, we received 144 comments from

schools about particular students. In 41 cases, we

randomly selected replacement students because the

children initially selected had left the school between

the time the roll was provided and the start of the

assessment programme in the school, or were expected

to be away throughout the assessment week, or had

been included in the roll by mistake. The remaining

103 comments concerned children with special needs.

Each such child was discussed with the school and

a decision agreed. Fifteen students were replaced

because they were very recent immigrants or overseas

students who had extremely limited English language

skills. Nine students were replaced because they had

disabilities or other problems of such seriousness that

it was agreed that the students would be placed at risk if

they participated. Participation was agreed upon for the

remaining 79 students, but a special note was prepared

to give additional guidance to the teachers who would

assess them.

In the corresponding operation at year 4 level, we

received 142 comments from schools about particular

students. Eighteen students were changed because

the lists originally supplied were incorrect (students

from other than year 4 included). Thirty-three students

originally selected were replaced because they had left

the school or were expected to be away throughout

the assessment week. Twelve students were replaced

because of their NESB status and very limited English.

Nineteen students were replaced because they had

disabilities or other problems of such seriousness the

students appeared to be at risk if they participated.

Special notes for the assessing teachers were made

about 60 children retained in the sample.

Communication with parents

Following these discussions with the school, Project

staff prepared letters to all of the parents, including a

copy of the NEMP brochure, and asked the schools to

address the letters and mail them. Parents were told

they could obtain further information from Project staff

(using an 0800 number) or their school principal, and

advised that they had the right to ask that their child be

excluded from the assessment.

At the year 8 level, we received a number of phone

calls including several from students wanting more

information about what would be involved. Five children

were replaced as a result of these contacts, one at the

child’s request and four at the parents’ request.

At the year 4 level we also received several phone

calls from parents. Some wanted details confirmed or

explained (notably about reasons for selection). One

child was replaced at his parents’ request, because they

were worried about his missing classes.

Practical arrangement with schools

On the basis of preferences expressed by the schools, we

then allocated each school to one of the five assessment

weeks available and gave them contact information for

the two teachers who would come to the school for a

week to conduct the assessments. We also provided

information about the assessment schedule and the

space and furniture requirements, offering to pay for

hire of a nearby facility if the school was too crowded to

accommodate the assessment programme. This proved

necessary in several cases.

Results of the sampling process

As a result of the considerable care taken, and the

attractiveness of the assessment arrangements to schools

and children, the attrition from the initial sample was

quite low. Only about two percent of selected schools

did not participate, and less than two percent of the

originally sampled children had to be replaced for

reasons other than their transfer to another school or

planned absence for the assessment week. The sample

can be regarded as very representative of the population

from which it was chosen (all children in New Zealand

schools at the two class levels except the one to two

percent in special schools or schools with less than four

year 4 or year 8 children).

Of course, not all the children in the samples actually

could be assessed. Eleven year 8 students, 26 year 4

students, and 5 Māori immersion students left school

at short notice and could not be replaced. Parents

withdrew 2 year 8 students and 1 year 4 student too

late to be replaced. One year 4 student was found to

be in a Māori immersion programme, and should not

be assessed in English. A further 12 year 8 students,

16 year 4 students, and 7 Māori immersion students

were absent from school throughout the assessment

week. Some others were absent from school for some

of their assessment sessions, and a small percentage of

performances were lost because of malfunctions in the

video recording process. Some of the students ran out of

time to complete the schedules of tasks. Nevertheless,

for many tasks over 95 percent of the sampled students

were assessed. No task had less than 90 percent of the

sampled students assessed. Given the complexity of the

Project, this is a very acceptable level of participation.


46 NEMP Report 25: Listening and Viewing 2002

Composition of the sample

Because of the sampling approach used, regions were

fairly represented in the sample, in approximate proportion

to the number of school children in the regions.

Region

Percentages of students from each region

region % year 4 sample % year 8 sample

Northland 4.2 4.2

Auckland 32.5 31.6

Waikato 10.0 10.0

Bay of Plenty/Poverty Bay 8.3 7.5

Hawkes Bay 4.2 4.2

Taranaki 2.5 3.3

Wanganui/Manawatu 5.8 5.8

Wellington/Wairarapa 10.8 10.8

Nelson/Marlborough/West Coast 4.2 4.2

Canterbury 10.8 11.7

Otago 4.2 4.2

Southland 2.5 2.5

Demography

demographic variables:

p e r c e n t a g e s o f s t u d e n t s in e a c h c a t e g o r y

variable category % year 4 sample % year 8 sample

Gender Male 52 51

Female 48 49

Ethnicity Non-Māori 79 81

Māori 21 19

Geographic Zone Greater Auckland 32 32

Other North Island 46 46

South Island 22 22

Community Size < 10,000 18 13

10,000 – 100,000 24 28

> 100,000 58 59

School SES Index Bottom 30 percent 31 27

Middle 40 percent 40 39

Top 30 percent 29 34

Size of School < 20 y4 students 16

20 – 35 y4 students 21

> 35 y4 students 63

150 y8 students 48

Type of School Full Primary 33

Intermediate 53

Other (not analysed) 14


A key purpose of language is communication.

Language allows us to share knowledge,

experiences, information, feelings and ideas.

Our day to day transactions of personal and

social activity rely heavily on language and

its communicative powers, and much of the

learning that takes place throughout the school

curriculum is inescapably language dependent.

Listening and viewing can be inseparable dimensions in

the receiving and understanding of messages. Effective

listening requires abilities to obtain information and

respond appropriately, to establish relationships

and interact with others, and to reflect upon ideas,

experiences and opinions. Viewing involves the

development of such skills as recognising the interaction

between words and images, and thinking critically about

the intentions, effects and impact of visual messages.

ISSN 1174-0000

ISBN 1-877182-39-7

National monitoring provides a “snapshot” of what

New Zealand children can do at two levels in primary

and intermediate schools: ages 8–9 and ages 12–13.

The main purposes for national monitoring are:

• to meet public accountability and information

requirements by identifying and reporting patterns

and trends in educational performance

• to provide high quality, detailed information which

policy makers, curriculum planners and educators

can use to debate and review educational practices

and resourcing.

LISTENING AND VIEWING a s s e s s m e n t r e s u l t s 2002 nemp

More magazines by this user
Similar magazines