MODUL PEMBINAAN ITEM TIMMS & PISA

TIMMS AND

PISA ITEM

CONSTRUCTION

MODULE

2020

IPGM/BPK

COMPILED BY :

Pn Nor Afizah bt Mohd Noor

Dr Ng Kok Fu

Dr Teong Mee Mee

Dr Jong Cherng Meei

Pn Kamariah bt Bujang

Pn Parimalarani a/p Sivasubramaniam

En Zulkifli bin Husin

Pn Norliyah bt Sulaiman

1

OBJECTIVES

2

Objectives

At the end of the course, participants

should be able to:

1. identify the criteria of higher-order

thinking skills items.

2. construct TIMMS and PISA items which

enhance higher-order thinking skills.

3. disseminate the course content

knowledge and skills to other

mathematics teachers.

3

WORKSHOP

IMPLEMENTATION

MODE

4

5

TIMMS

The Trends in International

Mathematics and Science Study

6

CONTENT OF TIMMS

TIMMS 2019 Mathematics

Assessment Framework

Sample Released TIMMS Items

The Process

7

TIMMS 2019 MATHEMATICS

ASSESSMENT FRAMEWORK

TIMMS the Trends in International Mathematics and Science Study, is an

international assessment of student achievement in mathematics and science at fourth and

eighth grades to enable countries around the world to make evidence-based decisions to

improve educational policies related to mathematics and science teaching and learning.

Dimensions

The assessment frameworks for TIMSS 2019 is organized around two dimensions:

● Content dimension, specifying the subject matter to be assessed (Table 1 & 2)

●

Cognitive dimension, specifying the thinking processes to be assessed

(Table 3 & 4)

Content Domains

Content Domains Topic Areas Percentage

Number

(50%)

Measurement and

Geometry

(30%)

Data

(20%)

Whole Numbers 25%

Expressions, simple equations, 15%

and relationships

Fractions and decimals 10%

Measurement 15%

Geometry 15%

Reading, interpreting, and

15%

representing data

Using data to solve problems 15%

Table 1: Target Percentages of the TIMSS 2019 Mathematics Assessment Devoted to

Content Domain of Grade 4

8

Content Domains

Number

(30%)

Algebra

(30%)

Geometry

(20%)

Data and probability

(20%)

Areas

Integers 10%

Fractions and decimals 10%

Ratio, proportion, and percent 10%

Expression, operations and

20%

equations

Relationship and functions 10%

Geometric shapes and

20%

measurements

Data 15%

Probability 5%


Content Domain of Grade 8

1. Number

(i) Whole Numbers

1. Demonstrate knowledge of place value (2-digit to 6-digit numbers); represent whole numbers

with words, diagrams, number lines, or symbols; order numbers.

2. Add and subtract (up to 4-digit numbers), including computation in simple contextual

problems.

3. Multiply (up to 3-digit by 1-digit and 2-digit by 2-digit numbers) and divide (up to 3-digit by 1-

digit numbers), including computation in simple contextual problems.

4. Solve problems involving odd and even numbers, multiples and factors of numbers, rounding

numbers (up to the nearest ten thousand), and making estimates.

5. Combine two or more properties of numbers or operations to solve problems in context.

(ii) Expressions, Simple Equations, and Relationships

1. Find the missing number or operation in a number sentence (e.g., 17 + w = 29).

2. Identify or write expressions or number sentences to represent problem situations that may

involve unknowns.

3. Identify and use relationships in a well-defined pattern (e.g., describe the relationship

between adjacent terms and generate pairs of whole numbers given a rule).

(iii) Fractions and Decimals

1. Recognize fractions as parts of wholes or collections; represent fractions using words,

numbers, or models; compare and order simple fractions; add and subtract simple fractions,

including those set in problem situations. (Fractions may have denominators of 2, 3, 4, 5, 6,

8, 10, 12, or 100.)

9

2. Demonstrate knowledge of decimal place value including representing decimals using

words, numbers, or models; compare, order, and round decimals; add and subtract

decimals, including those set in problem situations. (Decimals may have one or two decimal

places, allowing for computations with money.)

2. Measurement and Geometry

(i) Measurement

1. Measure and estimate lengths (millimeters, centimeters, meters, kilometers); solve problems

involving lengths.

2. Solve problems involving mass (gram and kilogram), volume (milliliter and liter), and time

(minutes and hours); identify appropriate types and sizes of units and read scales.

3. Solve problems involving perimeters of polygons, areas of rectangles, areas of shapes

covered with squares or partial squares, and volumes filled with cubes.

(ii) Geometry

1. Identify and draw parallel and perpendicular lines; identify and draw right angles and angles

smaller or larger than a right angle; compare angles by size.

2. Use elementary properties, including line and rotational symmetry, to describe, compare,

and create common two-dimensional shapes (circles, triangles, quadrilaterals, and other

polygons).

3. Use elementary properties to describe and compare three-dimensional shapes (cubes,

rectangular solids, cones, cylinders, and spheres) and relate these with their twodimensional

representations.

3. Data

(i) Reading, Interpreting, and Representing Data

1. Read and interpret data from tables, pictographs, bar graphs, line graphs, and pie charts.

2. Organize and represent data to help answer questions.

(ii) Using Data to Solve Problems

1. Use data to answer questions that go beyond directly reading data displays (e.g., solve

problems and perform computations using data, combine data from two or more sources,

draw conclusions based on data).

( Students will be permitted to use calculators, they may bring their own

calculator. In eTIMMS, students will have access to a calculator provided on-screen and

will not be permitted to bring their own).

10

Cognitive Domains

TIMSS assesses a range of problem solving situations within mathematics, with about twothirds

of the items requiring students to use applying and reasoning skills.

Cognitive Domains Percentage Focus

Knowing 40%

Applying 40%

Reasoning 20%

Facts

Concepts

Procedures

Application of knowledge

Conceptual understanding

Non-routine problems to encompass

unfamiliar situations, complex contexts, and

multistep problems.


Cognitive Domains of Grade 4

Cognitive domains Percentages Focus

Knowing 35% Facts

Concepts

Procedures

Applying 40% Representations

Problem solving

Reasoning 25% Making conjectures

Making logical deductions


Cognitive Domains of Grade 8

11

KNOWING DOMAIN

Recall

Recognize

Recall definitions, terminology, number properties, units of

measurement, geometric properties, and notation (e.g., a × b = ab, a

+ a + a = 3a).

Recognize numbers, expressions, quantities, and shapes. Recognize

entities that are mathematically equivalent (e.g., equivalent familiar

fractions, decimals, and percents; different orientations of simple

geometric figures).

Classify/Order Classify numbers, expressions, quantities, and shapes by common

properties.

Compute

Retrieve

Measure

Carry out algorithmic procedures for +, –, ×, ÷, or a combination of

these with whole numbers, fractions, decimals, and integers. Carry

out straightforward algebraic procedures.

Retrieve information from graphs, tables, texts, or other sources.

Use measuring instruments; and choose appropriate units of

measurement.

Determine

APPLYING DOMAIN

Determine efficient/appropriate operations, strategies, and tools for

solving problems for which there are commonly used methods of

solution.

Represent/Mode

l

Implement

Display data in tables or graphs; create equations, inequalities,

geometric figures, or diagrams that model problem situations; and

generate equivalent representations for a given mathematical entity

or relationship.

Implement strategies and operations to solve problems involving

familiar mathematical concepts and procedures.

12

Analyze

Integrate/

Synthesize

Evaluate

Draw

Conclusions

Generalize

Justify

REASONING DOMAIN

Determine, describe, or use relationships among numbers,

expressions, quantities, and shapes

Link different elements of knowledge, related representations, and

procedures to solve problems.

Evaluate alternative problem solving strategies and solutions

Make valid inferences on the basis of information and evidence

Make statements that represent relationships in more general and

more widely applicable terms

Provide mathematical arguments to support a strategy or solution

13

SAMPLE RELEASED TIMMS ITEMS

Released Items for Grade 8

1. Place the four digits 3, 5, 7 and 9 into the boxes below in the positions that would give

the greatest result when the two numbers are multiplied.

×

2.

Content domain:Number

Topic area:Whole Numbers

Cognitive domain:Reasoning

Maximum point:1

Key:See scoring guide

Code Correct Response

10 93 × 75 or 75× 93

Incorrect Response

70 95 × 73 or 73× 95

71 97 × 53 or 53× 97

79 Other incorrect (including crossed out, erased, stray marks, illegible,

or off task)

Nonresponse

99 Blank

What is the sum of all the interior angles of pentagon ABCDE? Show your work.

14

Content domain: Geometry

Topic Area: Geometric Shapes

Cognitive Domain:Reasoning

Maximum point:2

Key: See Scoring guide

Code Correct response

10 12

Incorrect Response

79 Incorrect (including crossed out, erased, stray marks, illegible, or off task)

Non Response

99 Blank

3.

Content domain: Number

Topic Area: Fractions and Decimals

Question intent: To find location of N on number line

Cognitive Domain: Reasoning

Maximum point:1

Key: D

15

4.

Lines m and n are parallel.

What is the value of b?

Answer:______________

Content domain: Geometry

Topic Area: Geometric Shapes

Cognitive Domain:Reasoning

Maximum point:1

Key: See scoring guide


10 50

Incorrect Response

79 Incorrect (including crossed out, erased, stray marks, illegible, or off

task)

Non Response

99 Blank

a + b = 25

What is the value of 2a + 2b + 4?

Content domain: Algebra

Topic Area: Equations/Formulas and functions

Cognitive Domain:Applying

Maximum point:1


16


10 54

Incorrect Response

70 104


task)

Non Response

99 Blank

5. 480 students were asked to name their favourite sport. The results are shown in this table.

Sport

Number of students

Hockey 60

Football 180

Tennis 120

Basketball 120

Use the information in the table to complete and label this pie chart.

Content domain: Data and chance

Topic Area: Data Organization and Representation

Cognitive Domain:Applying

Maximum point:2



20 Hockey 1 , Football 3 ,Tennis, Basketball both 1 , and labels all correct

8 8 4

Partially Correct Response

10 All sectors correct size but labeling incomplete or missing

Incorrect Response


task)

Non Response

99 Blank

17

Released Items for Grade 4

1. Here is a calendar for December.

DECEMBER

S M T W T F S

1 2 3 4

5 6 7 8 9 10 11

12 13 14 15 16 17 18

19 20 21 22 23 24 25

26 27 28 29 30 31

Mary’s birthday is on Thursday, December 2. She is going on a trip exactly 3 weeks later.

On what day will she go on the trip?

A. December 16 th

B. December 21 st

C. December 23 rd

D. December 30 th

Content Domain

Main Topic

Cognitive Domain

Key

Measurement

Tools, techniques, formulas

Solving Routine Problems

C

18

2. One centimeter on the map represents 8 kilometers on the land.

How far apart are Oxford and Smithville on the land?

A 4km

B 16km

C 35km

D 50km

Content Domain

Main Topic

Cognitive

Domain

Key

Measurement

Tools, techniques, formulas

Reasoning

C

3. Shade 1 of the large triangle.

2

Content Domain: Number

Topic Area: Fractions and Decimals

Cognitive Domain: Applying

Maximum Points:1


19

Number of Students

Correct Response

10 Any 2 small triangles shaded

11 Half of the triangle shaded in a way other than code 10

Incorrect Response

70 1 triangle shaded

79 Other incorrect (including crossed out, erased, stray marks, illegible, or

off task)

Non Response

99 Blank

4. Mr. Johnson asked the students in his school about their favorite subject.

This pie chart shows how many students liked each of 5 subjects.

Favorite Subject

Music

Gym

Maths

History

Science

Which graph shows the same information as the pie chart?

40

A

30

20

10

0

Maths Science History Gym Music

20

Number of Students

Number of Students

35

30

25

B

20

15

10

5

0


Axis Title

C

D

35

30

25

20

15

10

5

0


35

30

25

20

15

10

5


0Number of Students

Content Domain: Data Display

Topic Area: Organizing and Representing

Cognitive Domain: Reasoning

Maximum Points:1

Key: A

21

THE PROCESS

Sample 1 ( PRIMARY STANDARD) -

(i) Original Item

Domain:

Mathematics Literacy

Unit Title: Geometry

Scenario :

Proposed stimulus 1: Great Wall

Amir’s father wants to construct a rectangular wall by using 800 bricks.

The diagram below shows a rectangular wall.

22

Item 1 : GREAT WALL

What is the area of the wall?

A

B

8 m + 2 m

8 m × 2m

C 8 m × 2 m × 8

D

2 × 8 m × 8 m

ITEM SPECIFICATION

Question Intent

Understand the formula of an area

Item Code

Content Demand

Cognitive Domain

Measurement

Knowing

Cognitive Demand

Easy

Item Format

Multiple choice

23

SCORING GUIDE

Full Credit

Code 1: B

No Credit

Code 0: Other responses

Code 9: Missing

Item 2 : GREAT WALL

A brick cost RM0.45. How much does Amir’s father needs to pay?

A

B

C

D

RM360

RM324

RM270

RM243

ITEM SPECIFICATION

Question Intent

Calculate the total price for 800 bricks.

Item Code

Content Demand

Cognitive Domain

Number

Applying

Cognitive Demand

Item Format

Average

Multiple choice

SCORING GUIDE

Full Credit

Code 1: A

No Credit

24

Code 0: Other responses.

Code 9: Missing.

Item 3 : GREAT WALL

Amir’s father wants to put a tile with the size of surface 20 cm × 10 cm on the wall with area

16 m 2 . How many tiles he needs?

A 400

B 600

C 800

D 1000

ITEM SPECIFICATION

Question Intent

Understand the formula for an area of a wall, can

convert cm to m and calculate the bricks needed

Item Code

Content Demand

Cognitive Domain

Cognitive Demand

Item Format

SCORING GUIDE

Measurement

Applying

Difficult

Multiple choice

Full Credit

Code 1: C

No Credit


Code 9: Missing.

Item 4 : GREAT WALL

The table below shows the prices of two grades of bricks, A and B. Amir’s father has RM600

and he needs 800 pieces of bricks to build a wall. He just can choose one type of the bricks.

Which grade of brick should Amir’s father choose? Explain your answer.

25

Grade A B

Cost RM0.85 RM0.45

ITEM SPECIFICATION

Question Intent

Item Code

Analyze and compare two grades of bricks and choose

the bricks needed.

Content Demand

Cognitive Domain

Cognitive Demand

Item Format

Number

Applying

Difficult

Constructed Response

SCORING GUIDE

Full Credit

Code 1:

· Type B because the cost is less than RM600

· Type B because the cost for type A is more expensive and exceed the

budget.

No Credit

Code 9: Missing.


26

(ii) Reviewed Item

STIMULUS

Unit title Domain Framewor

k

ACER review summary:

suggested action

Great Wall Mathematics TIMSS Implement suggested

modifications

Comments/Modifications

Stimulus:

review of

general

and

specific

criteria

This is a nice, clear stimulus. A few strong items have been created

from this.

Suggestions to strengthen the stimulus:

●

Make the item about Amir not Amir’s father. This

simplifies the context and the reading load for the items.

●

Make the second sentence more direct

Suggested revision:

“Amir wants to construct a rectangular wall using 800 bricks.

This is the plan for the wall.”

Item 1: Great Wall MC

ACER

review Item

ID

Domain

Cognitive

Process/Skill

ACER

Estimated

item

Difficulty

ACER review

summary:

suggested

action

M027_T2 Mathematics TIMSS:

Knowing

Easy

Implement

suggested

modifications

27

Item


Entire Item

This is a good first item for this stimulus.

Stem

(Question)

Option A

Good

Option B

Key

Option C

For consistency ensure all numbers include unit eg. 8 m x 2 m x 8 m

Alternate option, 8 m – 2 m

Option D

For consistency ensure all numbers include unit eg. 2 m x 8 m x 8 m

Alternate option, 8 m ÷ 2 m

Scoring

guide


suggested action

Implement suggested

modifications


Question

intent

statement

Suggested revision “Recognises the formula for area.”

Question

intent

statement

matches

the correct

response?

The use of the word understand implies an ability to apply the formula.

The intention of this item is knowing the formula.

28

If MC, the

Key is

correct?

If CMC, are

the scoring

categories

appropriate?

Yes

Item 2 : Great Wall MC

ACER

review

Item ID

Domain

Cognitive

Process/Skill

ACER

Estimated

item

Difficulty

ACER review

summary:

suggested

action


Applying

Average

Implement

suggested

modifications

Item


Entire Item

A nice contextualised item.

Is the price used sourced from somewhere? Please check the price and

include the source and obtain permission to use. Accuracy for pricing is

important.

Stem

(Question)

Suggested revision “One brick costs RM0.45. How much did Amir’s pay

for his bricks?”

It is clearer to state one instead of a. Amir already has the bricks so the

question should be in past tense.

Option A

Key

Option B

29

Option C

Option D

Scoring

guide


suggested action

Implement suggested

modifications


Question

intent

statement

Question

intent

statement

matches the

correct

response?

If MC, the

Key is

correct?

If CMC, are

the scoring

categories

appropriate?

Good

Yes, clearly worded

Yes

What is the reasoning behind these option? A good way to develop

options is to consider common mistakes student will make when

attempting to answer a question. For example, 800 x RM0.4 instead of

RM0.45 so a good option would be RM320

Item 3 : Great Wall MC

ACER

review Item

ID

Domain

Cognitive

Process/Skill

ACER

Estimated

item

Difficulty

ACER review

summary:

suggested

action

30

M029_T2

Mathemat

ics

TIMSS:

Applying

Difficult

Implement

suggested

modifications

Item


Entire Item

The item intention is solid. To get the true value out of the item it

would work better as a short answer, open response item. The reason

for this is that there are multiple steps and it is worth allowing for

partial credit. Depending on how this item is used, it will also inform

teachers on the approach students are using to solve the problem.

The suggested revision clarifies context and reduces the reading load.

The area has been removed from the stem as the student should use

the information in the stimulus to calculate the area.

Stem

(Question)

Suggested revision,

“Amir wants to use tiles to cover the surface area of the wall.

He chooses 20 cm x 10 cm tiles.

How many tiles will Amir need to cover this area?”

Option A

Option B

Option C

Key

Option D

Scoring guide


suggested action

31

Implement suggested

modifications


Question

intent

statement

The student doesn’t have to convert the units of measure to correctly

answer this question. But they do need to have a solid understanding

of the relationship between the units of measure.

Suggested revision: “Understand the relationship between units of

measure and apply this to a real world problem about area.”

Question intent

statement

matches the

correct

response?

If MC, the Key

is correct?

If CMC, are the

scoring

categories

appropriate?

Code 2: 800

Code 1: May show correct process with some calculation errors.


Code 9: Missing.

32

Examples of

student

responses for

each scoring

category

Code 2: 16 ÷ 0.02 = 800

OR A student may use a diagram like this indicating how the tile may

be laid and then calculate how many tiles will fit. As below.

Code 1: Miscalculation of area 8 x 2 = 18 then 18 ÷ 0.02 = 900

Item 4 : Great Wall CR

ACER review

Item ID

Domain

Cognitive

Process/Skil

l

ACER

Estimated

item

Difficulty

ACER review

summary:

suggested action


Applying

Easy

Reject

Item


33

Entire Item

The intention behind this item is a good one. However, the content

chosen is dependent on another item within the unit. Amir has

already purchased the bricks for RM0.45 per brick in an earlier

item.

A way to incorporate the skills of analysing and evaluating could

be to consider how much grout would be needed to build the wall

and attach the tiles. The item could include different sized bags of

grout and a budget. For example,

Amir needs to use 70 kg of grout to hold the bricks and tiles in

place.

He wants to spend as little money as possible.

Which bags should Amir buy?

B

a

g

1

0

k

g

2

0

k

g

5

0

k

g

C

o

s

t

R

M

1

0

R

M

3

0

R

M

4

0

Stem

(Question)

Scoring guide

This item has been well worded. There are some minor grammar

issues but the concept of this item is well thought out and clearly

presented.


action

Implement suggested

modifications


34

Question intent

statement

This intention would be more accurate if both of the prices were

new.

Description for

each scoring

category

The scoring guide descriptions are good.

Examples of

student

responses for

each scoring

category

(iii) Final Item

Domain:

Matematik Tahun 5

Tajuk Unit:

‘Great Wall’

Senario

Cadangan Stimulus 1

Ayah Amir ingin membina sebuah dinding segi empat tepat dengan menggunakan 800 batu

bata. Rajah di bawah menunjukkan dinding berbentuk segi empat tepat.

2m

8m

35

Item 1 : ‘GREAT WALL’

Berapakah luas dinding tersebut?

A 8 m + 2 m

B 8 m × 2 m

C 8 m × 2 m × 8

D 2 × 8 m × 8 m

SPESIFIKASI ITEM

Tujuan soalan

Memahami formula luas

Kod Item

Domain Kandungan

Domain Kognitif

Aras Kognitif

Format Item

Ukuran dan Geometri

Pengetahuan

Mudah

Aneka Pilihan

PANDUAN PENSKORAN

Kredit Penuh

Kod 1: B

Tiada Kredit

Kod 0: Respon lain.

Kod 9: Tiada respon.

Item 2: ‘GREAT WALL’

Kos sebiji batu bata ialah RM0.45. Berapakah jumlah harga batu bata yang perlu dibayar

oleh ayah Amir?

A RM360

B RM324

C RM270

D RM243

36

SPESIFIKASI ITEM

Tujuan soalan

Kod Item

Domain Kandungan

Domain Kognitif

Aras Kognitif

Format Item

Mengira jumlah harga untuk 800 biji batu bata

Nombor

Aplikasi

Sederhana

Aneka Pilihan

PANDUAN PENSKORAN

Kredit Penuh

Kod 1: A

Tiada Kredit

Kod 0: Respon lain.


Item 3 : ‘GREAT WALL’

Ayah Amir ingin meletakkan jubin dengan saiz permukaan 20 cm × 10 cm di dinding yang

berkeluasan 16 m 2 . Berapakah bilangan jubin yang diperlukan?

A 400

B 600

C 800

D 1000

37

SPESIFIKASI ITEM

Tujuan soalan

Memahami formula luas dinding, boleh menukar unit cm

kepada m dan mengira bilangan jubin yang diperlukan.

Kod Item

Domain Kandungan

Domain Kognitif

Aras Kognitif

Format Item

Ukuran dan Geometri

Aplikasi

Sukar

Aneka Pilihan

PANDUAN PENSKORAN

Kredit Penuh

Kod 1: C

Tiada Kredit

Kod 0: Respon lain.


Item 4: ‘GREAT WALL’

Jadual di bawah menunjukkan harga dua jenis batu bata, A dan B. Ayah Amir mempunyai

RM600 dan memerlukan 800 biji batu bata untuk membina sebuah dinding. Beliau hanya

boleh memilih satu jenis batu bata. Batu bata yang manakah patut ayah Amir pilih? Jelaskan

jawapan anda.

Jenis A B

Kos RM0.85 RM0.45

38

SPESIFIKASI ITEM

Tujuan soalan

Menganalisis dan membanding dua gred batu bata dan

memilih bata yang diperlukan.

Kod Item

Domain Kandungan

Domain Kognitif

Aras Kognitif

Format Item

Nombor

Aplikasi

Sukar

Respon Konstruktif

PANDUAN PENSKORAN

Kredit Penuh

Kod 1:

● Jenis B kerana harga kos kurang daripada RM600

● Jenis B kerana harga kos jenis A lebih mahal dan melebihi anggaran

perbelanjaan.

Tiada Kredit

Kod 0: Respon lain.


39

Sample 2 ( Primary ) - Example : Social Media

(i) Original Item

Domain:

Mathematics Grade 5

Unit Title: Social Media

Scenario :

Proposed stimulus 1

The pie chart below shows the percentage of social media platforms in Kulim, Kedah

2018.

Item 1: SOCIAL MEDIA

Which is the most popular social media among the users?

40

ITEM SPECIFICATION

Question Intent

Identify the most users of social media from pie chart

Item Code

Content Domain

Cognitive Process

Cognitive Demand

Item Format

Data

Knowing

Easy


SCORING GUIDE

Full Credit

Code 1: Instagram

No Credit


Code 9: Missing.

Item 2 : SOCIAL MEDIA

State the percentage of Facebook users in fraction.

ITEM SPECIFICATION

Question Intent

Convert percentage into fraction

Item Code

Content Domain

Cognitive Process

Cognitive Demand

Item Format

Number

Knowing

Easy


41

SCORING GUIDE

Code 1:

No Credit


Code 9: Missing


In 2018, the total users of social media in Kulim is 90 000. Calculate the total users

of Twitter.

ITEM SPECIFICATION

Question Intent

Determine number of users

Item Code

Content Domain

Cognitive Process

Cognitive Demand

Item Format

Number

Applying

Average


SCORING GUIDE

Code 1:

· 20 700

· 23% × 90 000

No Credit


Code 9: Missing

42


Here are some statements about the chart.

Put an X to show whether each statement is true or false.

Statement True False

The total users of Facebook is 24 300.

The percentage difference between Twitter and

Facebook users is 3%.

The total number of Instagram and YouTube

users is 53 100.

ITEM SPECIFICATION

Question Intent

Determine the validity of three statements based

on given pie chart

Item Code

Content Domain

Cognitive Process

Cognitive Demand

Item Format

Data

Applying

Average

Complex Multiple Choice

SCORING GUIDE

Code 1: All three correct responses. True, False, True in that order.

No Credit

Code 0: Other responses. Code 9: Missing

43


What is the total percentage of YouTube and Facebook users?

A. 14%

B. 27%

C. 41%

D. 64%

ITEM SPECIFICATION

Question Intent

Determine the percentage of social media user

Item Code

Content Domain

Cognitive Process

Cognitive Demand

Item Format

Number

Knowing

Average

Multiple Choice

SCORING GUIDE

Code 1: C

No Credit


Code 9: Missing.

44

(ii) Reviewed Item

STIMULUS

Unit title Domain Framework ACER review summary:

suggested action

Social

Media

Mathematics TIMSS Implement suggested

modifications


Stimulus:

review of

general

and

specific

criteria

This is a well-planned and relevant contextual unit. Some revisions

are necessary to clarify the data presented in the stimulus.

●

●

Is this real data? If so please include the source. If it is

not real data, a fictitious name should be used.

Suggested rewording to explain what the nature of the

data

“The pie chart shows the percentage of the population that uses

social media platforms in Kulim, Kedah 2018.”

Item 1 Social Media CR

ACER

review

Item ID

Domain

Cognitive

Process/Skill

ACER

Estimated item

Difficulty

ACER review

summary:

suggested

action


Knowing

Easy

Implement

suggested

modifications

45

Item


Entire

Item

With the revisions to the stimulus, this makes for a nice first item. It is

clear and straight forward.

Stem

(Question

)

Scoring

guide


action

Accept without modification


Question

intent

statement

Descriptio

n for each

scoring

category

Examples

of student

responses

for each

scoring

category

Well done.

Good

NA

Item 2 Social Media CR

ACER

review

Item ID

Domain

Cognitive

Process/Skill

ACER

Estimated item

Difficulty

ACER review

summary:

suggested

action

46


Applying

Difficult

Implement

suggested

modifications

Item


Entire

Item

This is well worded and clear. As this is a two-step problem it is worth

allowing for partial credit as the maths involved is also difficult. The

student can then be awarded some marks for correct process.

The content domain for this item is Data, not number, as the student

has to extract information for the chart before calculating the response

and checking that it seems reasonable based on the data in the pie

chart.

Stem

(Question

)

Scoring

guide

Is the population of Kulim exactly 90 000? Please include the source. If

it is not exactly 90 000 consider adding a statement such as “close to

90 000”


action

Implement suggested

modifications


Question

intent

statement

The question intention should describe what the student needs to do to

correctly answer the question. In this case, calculate the percentage of

a large whole number.

Descriptio

n for each

scoring

category

Include partial credit:

Code 2: 20 700

Code 1: 0.23 x 90 000 with incorrect calculation


Code 9: Missing

47

Examples

of student

responses

for each

scoring

category

Code 1: 0.23 x 90 000 = 2070

Item 3 Social Media CMC

ACER review

Item ID

Domain

Cognitive

Process/Skill

ACER

Estimated

item Difficulty

ACER review

summary:

suggested

action


Applying

Average

Accept without

modification

Item


Entire Item

This is a well worded item, which draws on different elements of the

data.

Stem

(Question)

Option A

Option B

Option C

Option D

Scoring guide


suggested action

48

Implement suggested

modifications


Question

intent

statement

Question

intent

statement

matches the

correct

response?

If MC, the

Key is

correct?

If CMC, are

the scoring

categories

appropriate?

Well written.

Yes, well done.

Incorrect key. The key should be True, False, False

Item 4 Social Media MC

ACER review

Item ID

Domain

Cognitive

Process/Skill

ACER

Estimated

item

Difficulty

ACER review

summary:

suggested action


Knowing

Easy

Implement

suggested

modifications

Item


49

Entire Item

This is a nice simple item. It would however, be better placed at the

beginning of the unit. The last item in a unit should be the most

challenging. The reason for this is that the student has had time to

become familiar with the context of the unit and is well prepared to

dig deeper into the skills and contexts at the end of a unit.

Again this is a Data item.

Stem

(Question)

Option A

Option B

Option C

Key

Option D

Scoring guide


suggested action

Implement suggested

modifications


Question intent

statement

A more accurate statement would be “Calculate the total

percentage of two groups within a pie chart.”

Question intent

statement

matches the

correct

response?

No

50

If MC, the Key is

correct?

If CMC, are the

scoring

categories

appropriate?

What is the reasoning behind these option? A good way to develop

options is to consider common mistakes student will make when

attempting to answer a question. For example, calculating

Facebook and Twitter instead of Facebook and YouTube therefore

50% would be a good option.

(iii) Final Item

Domain:

Matematik Tahun 5

Tajuk Unit:

Media Sosial

Senario:

Cadangan Stimulus 1

Carta pai di bawah menunjukkan peratus platform media sosial di Kulim, Kedah 2018.

51

Item 1: MEDIA SOSIAL

Media sosial yang manakah paling popular dikalangan pengguna?

SPESIFIKASI ITEM

Tujuan soalan

Mengenalpasti pengguna media sosial tertinggi

daripada carta pai

Kod Item

Domain Kandungan

Domain Kognitif

Aras Kognitif

Format Item

Data

Pengetahuan

Mudah

Respon Konstruktif

PANDUAN PENSKORAN

Kredit Penuh

Kod 1: Instagram

Tiada Kredit

Kod 0: Respon lain.


Item 2 : MEDIA SOSIAL

Nyatakan peratus pengguna Facebook dalam pecahan.

SPESIFIKASI ITEM

Tujuan soalan

Menukarkan peratus kepada

pecahan

Kod Item

Domain Kandungan

Domain Kognitif

Aras Kognitif

Format Item

Nombor

Pengetahuan

Mudah

Respon Konstruktif

52

PANDUAN PENSKORAN

Kredit Penuh

Kod 1: 27

100

Tiada Kredit

Kod 0: Respon lain.



Dalam tahun 2018, pengguna media sosial di Kulim adalah sebanyak 90 000. Kira

jumlah pengguna Twitter.

SPESIFIKASI ITEM

Tujuan soalan

Menentukan bilangan pengguna

Kod Item

Content Domain

Domain Kandungan

Domain Kognitif

Aras Kognitif

Format Item

Nombor

Aplikasi

Sederhana

Respon Konstruktif

PANDUAN PENSKORAN

Kredit Penuh

Kod 1:

● 20 700

● 23% × 90 000

Tiada Kredit

Kod 0: Respon lain.


53


Berdasarkan carta pai tersebut, bulatkan “Betul” atau “Salah” bagi setiap penyataan.

Jumlah pengguna Facebook ialah 24 300.

Perbezaan peratus antara pengguna Twitter dan

Facebook ialah 3%.

Jumlah pengguna Instagram dan YouTube ialah 53

100.

Betul / Salah

Betul / Salah

Betul / Salah

SPESIFIKASI ITEM

Tujuan soalan

Menentukan kesahan tiga penyataan

berdasarkan carta pai yang diberi

Kod Item

Domain Kandungan

Domain Kognitif

Aras Kognitif

Format Item

Data

Aplikasi

Sederhana

Aneka Pilihan Kompleks

PANDUAN PENSKORAN

Kredit Penuh

Kod 1: Semua jawapan betul. Betul, Salah, Betul mengikut urutan.

Tiada Kredit

Kod 0: Respon lain.


Item 5 : MEDIA SOSIAL

Berapakah jumlah peratus pengguna YouTube dan Facebook?

A. 14%

B. 27%

C. 41%

D. 64%

54

SPESIFIKASI ITEM

Tujuan soalan

Menentukan peratus pengguna media social

Kod Item

Domain Kandungan

Domain Kognitif

Aras Kognitif

Format Item

Nombor

Pengetahuan

Sederhana

Aneka Pilihan

PANDUAN PENSKORAN

Kredit Penuh

Kod 1: C

Tiada Kredit

Kod 0: Respon lain.


Sample 3 ( Primary ) - Example : Geometry

(i) Original Item

Domain:

Mathematics Year 5

Unit Title:

Colour your fractions

Scenario 1:

Proposed stimulus 1

1 hectare =

100metres 2

Samad owns a one-hectare plot of land.

He has divided his land into smaller plots and colour-coded the plots for easy identification.

He has built his house on the blue plot.

He plans to plant rambutan trees on the grey plots.

55

He wishes to give away all the yellow and red plots to his eldest son.

He wants to sell the green plot to his neighbour.

Item 1 :

What fraction of one-hectare does Samad’s house lie on?

1

A

32

B

C

D

1

16

1

8

1

4

ITEM SPECIFICATION

Question Intent

Identifying proper fractions as part of a whole.

Item Code

Content

Cognitive Domain

Cognitive Demand

Numbers

Knowledge

Easy

Context

Item Format

Multiple Choice

SCORING GUIDE

Answers

Full Credit

Code 1: D

No Credit


Code 9: Missing

56

Item 2 :

Samad gave away the yellow and red plots of his land to his eldest son as a wedding gift.

What fraction of one-hectare did his eldest son now own? Explain your answer.

ITEM SPECIFICATION

Question Intent

Item Code

Content

Cognitive Domain

Cognitive Demand

Context

Item Format

Addition of up to three proper fractions.

Subtraction of fractions.

Numbers

Reasoning

Moderate


SCORING GUIDE

Answers

Full Credit

Code 1:

● Each small square = 1 16 .

Yellow plot is equal to two small squares = 1 8 .

Red plots are equal to one and a half squares = 1 16 + 1 32 .

1

Therefore + 1 + 1 = 7 or

8 16 32 32

● Using reasoning, move the red plot to the right lower quadrant and the grey plot to

the red plot area.

Red and yellow plot

1

4 − 1 32 = 7 32

No Credit

Code 0: Other incorrect responses

Code 9: Missing

represent:

57

Item 3:

If Samad’s land is worth RM120 000, how much is the green plot worth?

Explain your answer.

ITEM SPECIFICATION

Question Intent

Item Code

Content

Cognitive Domain

Cognitive Demand

Context

Item Format

Determine the value of a fraction of a quantity.

Numbers

Reasoning

Moderate


SCORING GUIDE

Answers

Full Credit

Code 1:

● By using reasoning :

Green plot consists of 1 1 small squares.

2

16 small squares = RM120 000

1 small square = RM 7500

1 1 small squares = RM 7500 + RM 3750

2

= RM 11 250.

● 1 hectare = RM120 000

1 1 small squares = 3 2 32

3

of 1 hectare = 3 X RM120 000

32 32

= RM11 250.

No Credit


Code 9: Missing

58

Item 4:

Samad wants to plant rambutans trees on the grey plots of land.

What part of one-hectare would be planted with rambutan trees. Explain your answer.

ITEM SPECIFICATION

Question Intent

Item Code

Content

Cognitive Domain

Cognitive Demand

Context

Item Format

Problem solving involving fractions.

Numbers

Reasoning

Moderate


SCORING GUIDE

Answers

Full Credit

Code 1:

1

● + 1 + 1 + 1 + 1 + 1 + 1 = 14

= 7 or

16 16 16 16 8 32 32 32 16

● Use reasoning to rearrange the coloured plots to obtain

So, 1 - ( 1 + 1 ) = 7 or

2 16 16

● Using reasoning: there are 7

grey square represents 1

Total grey plot = 7 16 .

16.

small grey squares, each small

No Credit


Code 9: Missing

59

Item 5:

Here are some statements about Samad’s land.



The percentage of Samad’s land that was sold is less

than 10%.

The percentage of the yellow and red plots are more

than 25%.

The percentage of the blue, green and yellow plots

altogether is more than 45% but less than 50 %.

ITEM SPECIFICATION

Question Intent

Item Code

Content

Cognitive Domain

Cognitive Demand

Context

Item Format

Calculating percentages.

Numbers

Application

Moderate


SCORING GUIDE

Answers

Full Credit


No Credit


Code 9: Missing

60

(ii) Reviewed Item

Domain:

Mathematics Year 5

Unit Title:


Scenario 1:

Proposed stimulus 1

Samad’s Land

1 hectare =

100metres 2

Samad owns a one-hectare plot of land.


He has built his house on the blue plot.

He plans to plant rambutan trees on the grey plots.

He wishes to give away all the yellow and red plots to his eldest son.

He wants to sell the green plot to his neighbour.

Item 1:

What fraction of one-hectare does the blue plot represent?

A

B

C

D

1

32

1

16

1

8

1

4

ITEM SPECIFICATION

Question Intent


Item Code

Content

Cognitive Domain

Numbers

Knowledge

61

Cognitive Demand

Easy

Context

Item Format

Multiple Choice

SCORING GUIDE

Answers

Full Credit

Code 1: D

No Credit


Code 9: Missing

Item 2:


What fraction of one-hectare the land does his son own? Explain your answer.

ITEM SPECIFICATION

Question Intent

Item Code

Content

Cognitive Domain

Cognitive Demand

Addition or subtraction of up to three proper fractions.

Subtraction of proper fractions.

Numbers

Reasoning

Moderate

Context

Item Format


SCORING GUIDE

Answers

Full Credit

Code 1:


62



1

Therefore + 1 + 1 = 7 8 16 32 32

or

● Using reasoning,

move the red plot to the right lower

quadrant and the grey plot

to the red plot area.

Red and yellow plot represent:

1

4 − 1 32 = 7 32

No Credit


Code 9: Missing

Item 3:

Samad wants to sell the green plot to his neighbour. If Samad’s original plot of land is

worth RM120 000, how much is the green plot worth?


ITEM SPECIFICATION

Question Intent

Item Code

Content

Cognitive Domain

Cognitive Demand


Numbers

Reasoning

Moderate

Context

Item Format


SCORING GUIDE

Answers

Full Credit

Code 1:



2


63


1 1 2

small squares = RM 7500 + RM 3750

= RM 11 250.

● 1 hectare = RM120 000

1 1 2 small squares = 3 32

3


32 32

= RM11 250.

No Credit


Code 9: Missing

Item 4:


What part of one-hectare percentage of his land would be planted with rambutan trees.


ITEM SPECIFICATION

Question Intent

Item Code

Content

Cognitive Domain

Cognitive Demand

Problem solving involving fractions

and percentages.

Numbers

Reasoning

Moderate

Context

Item Format


SCORING GUIDE

Answers

Full Credit

Code 1:

Find the fraction representing grey plot

● : 1 16 + 1 16 + 1 16 + 1 16 + 1 8 + 1 32 + 1 32 = 14

32 = 7 16

or

64

●

Use reasoning to rearrange the coloured plots to obtain

So, 1 - ( 1 + 1 ) = 7 or

2 16 16




16.

Then, calculate percentage of land planted with rambutan trees

= 7 16

× 100% = 43. 75%

No Credit


Code 9: Missing

Item 5:






than 10%.


than 25%.



ITEM SPECIFICATION

Question Intent

Item Code

Content

Cognitive Domain

Cognitive Demand


Numbers

Application

Moderate

Context

Item Format


65

SCORING GUIDE

Answers

Full Credit


No Credit


Code 9: Missing

(iii) Final item

Domain:

Mathematics Year 5

Unit Title:


Scenario 1:

Proposed stimulus 1

Samad’s Land

Samad owns a one-hectare

plot of land.


Item 1:

What fraction does the blue plot represent?

A

B

C

D

1

32

1

16

1

8

1

4

66

ITEM SPECIFICATION

Question Intent


Item Code

Content

Cognitive Domain

Cognitive Demand

Numbers

Knowledge

Easy

Context

Item Format

Multiple Choice

SCORING GUIDE

Answers

Full Credit

Code 1: D

No Credit


Code 9: Missing

Item 2 :


What fraction of the land does his son own? Explain your answer.

ITEM SPECIFICATION

Question Intent

Item Code

Content

Cognitive Domain

Cognitive Demand

Context

Item Format

Addition or subtraction of up to three proper fractions.

Subtraction of proper fractions.

Numbers

Reasoning

Moderate


67

SCORING GUIDE

Answers

Full Credit

Code 1:




1

Therefore + 1 + 1 = 7 or

8 16 32 32

● Using reasoning, move the red plot to the right lower quadrant and the grey plot to

the red plot area.

Red and yellow

represent:

1

4 − 1 32 = 7 32

No Credit


Code 9: Missing

Item 3:

plot

If Samad’s land is worth RM120 000, how much is the green plot worth?


ITEM SPECIFICATION

Question Intent

Item Code

Content

Cognitive Domain

Cognitive Demand


Numbers

Reasoning

Moderate

Context

Item Format


68

SCORING GUIDE

Answers

Full Credit

Code 1:



2



1 1 2

small squares = RM 7500 + RM 3750

● 1 hectare = RM120 000

= RM 11 250.

1 1 2 small squares = 3 32

3


32 32

= RM11 250.

No Credit


Code 9: Missing

Item 4:


What would be planted with rambutan trees. Explain your answer.

ITEM SPECIFICATION

Question Intent

Problem solving involving fractions and percentages.

Item Code

Content

Numbers

Cognitive Domain

Reasoning

Cognitive Demand

Moderate

Context

Item Format


SCORING GUIDE

Answers

Full Credit

Code 1:

Find the fraction representing grey plot

69

● : 1 16 + 1 16 + 1 16 + 1 16 + 1 8 + 1 32 + 1 32 = 14

●

32 = 7 16

Use reasoning to rearrange the coloured plots to obtain

or

So, 1 - ( 1 + 1 ) = 7 or

2 16 16




16.

Then, calculate percentage of land planted with rambutan trees

= 7 16

× 100% = 43. 75%

No Credit


Code 9: Missing


Item 5:





than 10%.


than 25%.



ITEM SPECIFICATION

Question Intent

Item Code

Content

Cognitive Domain

Cognitive Demand


Numbers

Application

Moderate

Context

Item Format


70

SCORING GUIDE

Answers

Full Credit


No Credit


Code 9: Missing

71

72

PISA

Programme for International

Student Assessment

73

CONTENTS OF PISA

PISA 2021 Mathematics Assessment

Framework

Sample Released PISA Items

The Process

74

PISA 2021 MATHEMATICS

ASSESSMENT FRAMEWORK

PISA is the OECD's Programme for International Student Assessment. PISA measures 15-

year-olds’ ability to use their reading, mathematics and science knowledge and skills to meet reallife

challenges.

https://pisa2021-maths.oecd.org/

Overview

The PISA 2021 mathematics framework defines the theoretical underpinnings of the PISA mathematics

assessment based on the fundamental concept of mathematical literacy, relating mathematical

reasoning and three processes of the problem-solving (mathematical modelling) cycle, namely to

formulate, to employ, and to interpret and evaluate. The framework describes how mathematical content

knowledge is organised into four content categories of Quantity, Uncertainty and Data, Change and

Relationships, and Space and Shape. It also describes four categories of contexts, namely personal,

occupational, societal and scientific contexts, in which students will face mathematical challenges. The

PISA assessment measures how effectively countries are preparing students to use mathematics in

every aspect of their personal, civic and professional lives, as part of their constructive, engaged and

reflective 21st Century citizenship.

What is Mathematical Literacy?

Mathematical literacy is an individual’s capacity to reason mathematically and to formulate, employ and

interpret mathematics to solve problems in a variety of real-world contexts. It includes concepts,

procedures, facts and tools to describe, explain and predict phenomena. It helps individuals know the

role that mathematics plays in the world and make the well-founded judgments and decisions needed

by constructive, engaged and reflective 21st Century citizens.

Mathematical Reasoning

The ability to reason logically and present arguments in honest and convincing ways is a skill that is

becoming increasingly important in today’s world. In mathematics, students learn that, with proper

reasoning and assumptions, they can arrive at trusted and impartial results in a wide variety of real-life

contexts. There are at least six key understandings that provide structure and support to mathematical

reasoning:

(a) understanding quantity, number systems and their algebraic properties;

(b) appreciating the power of abstraction and symbolic representation;

(c) seeing mathematical structures and their regularities;

(d) recognising functional relationships between quantities;

(e) using mathematical modelling as a lens onto the real world; and

(f) understanding variation as the heart of statistics.

75

(a) Quantity, number systems and their algebraic properties

The concept of quantity is conceptualised in mathematics by the concept of number systems and the

basic algebraic properties. It is also important to understand matters of representation (as symbols

involving numerals, as points on a number line or as geometric quantities) and how to move between

them; the ways in which these representations are affected by number systems; and the ways in which

algebraic properties of these systems are relevant for operating within the systems.

(b) Mathematics as a system based on abstraction and symbolic representation

Abstraction involves deliberately and selectively attending to structural similarities between objects and

constructing relationships between those objects based on these similarities. In school mathematics,

abstraction forms relationships between concrete objects, symbolic representations, and operations

including algorithms and mental models. Students use symbolic, graphical, numerical or geometric

representations to organise and communicate their mathematical thinking and to condense

mathematical meanings and processes into efficient algorithms. Representations are also a core

element of mathematical modelling, allowing students to abstract a simplified or idealised formulation

of a real-life problem.

(c) Mathematical structures and their regularities

Structure is intimately related to symbolic representation. Seeing structure is a way of finding and

remembering the meaning of an abstract representation. Being able to see structure is an important

conceptual aid to purely procedural knowledge. A robust sense of mathematical structure also supports

modelling. When the objects under study are not abstract mathematical objects, but rather objects from

the real world to be modelled by mathematics, then mathematical structure can guide the modelling.

76

(d) Functional relationships between quantities

Relationships between quantities can be expressed with equations, graphs, tables or verbal

descriptions. An important step in learning is to extract from these the notion of a function itself, as an

abstract object of which these are representations. The function can be represented in the form of a

graph, and reading a graph and coordinating the values on the axes has a dynamic or process aspect.

The graph provides a visual tool for understanding a function as a relationship between co-varying

quantities and for exploring the notion of a rate of change. Reading And the graph of a function is an

important tool.

(e) Mathematical modelling as a lens onto the real world

Models represent an ideal conceptualisation of a real-life or scientific phenomenon. They are

abstractions of reality. A model may be an approximation or working hypothesis concerning a

phenomenon or an intentional simplification. Mathematical models are formulated in mathematical

language and use a wide variety of mathematical tools and results (e.g. from arithmetic, algebra or

geometry). Therefore, they are used as ways of precisely defining the conceptualisation or theory of a

phenomenon, for analysing and evaluating data (does the model fit the data?) and for making

predictions.

(f) Variation at the heart of statistics

It is difficult to make generalisations in a world of large variation. In today’s world, people often deal with

situations by merely ignoring the variation and suggest sweeping generalisations that are often

misleading. Statistics is in many ways a search for patterns in a highly variable context: trying to find

the signal defining “truth” in the midst of a great deal of random noise. “Truth” is an estimate of truth set

in a probabilistic context, accompanied by an estimate of the error contained in the process. Ultimately,

the estimate is a set of plausible values.

The mathematical reasoning involves three main processes of formulating, employing, and interpreting

and evaluating.

1. Formulate

The word formulate refers to the ability of individuals to recognise and identify opportunities to use

mathematics and provide mathematical structure to a problem presented in some contextualised form.

In the process of formulating situations mathematically, individuals translate from a real-world setting to

the domain of mathematics and provide the real world problem with mathematical structure and

representations. They reason about and make sense of constraints and assumptions in the problem.

The process of formulating includes:

selecting an appropriate model from a list;

identifying the mathematical aspects of a problem situated in a real-life context and identifying

the significant variables;

recognising mathematical structure (including regularities, relationships and patterns) in

problems or situations;

simplifying a situation or problem in order to make it amenable to mathematical analysis;

identifying constraints and assumptions behind any mathematical modelling and simplifications

gleaned from the context;

representing a situation mathematically, using appropriate variables, symbols, diagrams and

standard models;

representing a problem in a different way, including organising it according to mathematical

concepts and making appropriate assumptions;

understanding and explaining the relationships between the context-specific language of a

problem and the symbolic and formal language needed to represent it mathematically;

translating a problem into mathematical language or a representation;

77

recognising aspects of a problem that correspond with known problems or mathematical

concepts, facts or procedures;

using technology (such as a spreadsheet or the list facility on a graphing calculator) to portray

a mathematical relationship inherent in a contextualised problem; and

creating an ordered series of (step-by-step) instructions for solving problems.

2. Employ

The word employ refers to the ability of individuals to apply mathematical concepts, facts, procedures

and reasoning to solve mathematically formulated problems to obtain mathematical conclusions. In the

process of employing mathematical concepts, facts, procedures and reasoning to solve problems,

individuals perform the mathematical procedures needed to derive results and find a mathematical

solution. They work on a model of the problem situation, establish regularities, identify connections

between mathematical entities and create mathematical arguments. The process of employing includes:

performing a simple calculation;

drawing a simple conclusion;

selecting an appropriate strategy from a list;

devising and implementing strategies for finding mathematical solutions;

using mathematical tools, including technology, to help find exact or approximate solutions;

applying mathematical facts, rules, algorithms and structures when finding solutions;

manipulating numbers, graphical and statistical data and information, algebraic expressions and

equations, and geometric representations;

making mathematical diagrams, graphs and constructions and extracting mathematical

information from them;

using and switching between different representations in the process of finding solutions;

making generalisations based on the results of applying mathematical procedures to find

solutions;

reflecting on mathematical arguments, and explaining and justifying mathematical results; and

evaluating the significance of observed or proposed patterns and regularities in data.

3. Interpret and Evaluate

The word interpret (and evaluate) focuses on the ability of individuals to reflect upon mathematical

solutions, results or conclusions and interpret them in the context of the real-life problem that initiated

the process. This involves translating mathematical solutions or reasoning back into the context of the

problem and determining whether the results are reasonable and make sense in the context of the

problem. The process of interpreting and evaluating includes:

interpreting information presented in graphical form and/or diagrams;

evaluating a mathematical outcome in terms of the context;

interpreting a mathematical result back into the real-world context;

evaluating the reasonableness of a mathematical solution in the context of a real-world problem;

understanding how the real world impacts the outcomes and calculations of a mathematical

procedure or model in order to make contextual judgments about how the results should be

adjusted or applied;

explaining why a mathematical result or conclusion does or does not make sense given the

context of a problem;

understanding the extent and limits of mathematical concepts and solutions;

critiquing and identifying the limits of the model used to solve a problem; and

using mathematical thinking and computational thinking to make predictions, to provide

evidence for arguments, and to test and compare proposed solutions.

78

Content Knowledge

To reason mathematically and to solve problems and interpret situations in personal, occupational,

societal and scientific contexts, individuals need to draw upon certain mathematical knowledge and

understanding. The four content areas are Quantity, Uncertainty and Data, Change and Relationships,

and Space and Shape.

1. Quantity

Quantification is a primary method for describing and measuring a vast set of attributes of aspects of

the world. It allows for the modelling of situations, for the examination of change and relationships, for

the description and manipulation of space and shape, for organising and interpreting data, and for the

measurement and assessment of uncertainty.

2. Uncertainty and Data

Uncertainty is a phenomenon at the heart of the mathematical analysis of many problem situations.

Probability, statistics, and techniques of data representation and description have been established to

deal with it. This category includes recognising variation in processes, having a sense of the

quantification of that variation, acknowledging uncertainty and error in measurement, and knowing

about chance. It also includes forming, interpreting and evaluating conclusions drawn in situations

where uncertainty is central. Quantification is a primary method for describing and measuring a vast set

of attributes of aspects of the world.

3. Change and Relationships

The natural and designed worlds display a multitude of temporary and permanent relationships among

objects and circumstances, where changes occur within systems of interrelated objects or in

circumstances where the elements influence one another. Being literate about change and relationships

involves understanding fundamental types of change and recognising when they occur in order to use

suitable mathematical models to describe and predict change. This means modelling the change and

the relationships with appropriate functions and equations, as well as creating, interpreting and

translating among symbolic and graphical representations of relationships.

4. Space and Shape

Space and shape encompass a wide range of phenomena that are encountered everywhere in our

visual and physical world: patterns, properties of objects, positions and orientations, representations of

objects, decoding and encoding of visual information, and navigation and dynamic interaction with real

shapes as well as with representations. Geometry serves as an essential foundation for space and

shape, but the category extends beyond traditional geometry in content, meaning and method, drawing

on elements of other mathematical areas such as spatial visualisation, measurement and algebra.

Contexts

An important aspect of mathematical literacy is that mathematics is used to solve a problem set in a

context. The context is the aspect of an individual’s world in which the problems are placed. The choice

of appropriate mathematical strategies and representations is often dependent on the context in which

a problem arises. For PISA, it is important that a wide variety of contexts are used.

79

Personal

Problems classified in the personal context category focus on activities of one’s self, one’s family or

one’s peer group which include food preparation, shopping, games, personal health, personal

transportation, sports, travel, personal scheduling, personal finance and others.

Occupational

Problems classified in the occupational context category are centred on the world of work. Items

categorised as occupational may involve such things as measuring, costing and ordering materials for

building, payroll/accounting, quality control, scheduling/inventory, design/architecture, and job-related

decision-making. Occupational contexts may relate to any level of the workforce, from unskilled work to

the highest levels of professional work.

Societal

Problems classified in the societal context category focus on one’s community, whether local, national

or global. They may involve such things as voting systems, public transport, government, public policies,

demographics, advertising, national statistics and economics. Although individuals are involved in all of

these things in a personal way, in the societal context category, the focus of problems is on the

community perspective.

Scientific

Problems classified in the scientific category relate to the application of mathematics to the natural world

and issues and topics related to science and technology. Particular contexts might include such areas

as weather or climate, ecology, medicine, space science, genetics, measurement and the world of

mathematics itself.

21st Century Skills

Some of the key 21st Century skills are critical thinking, creativity, research and inquiry, self-direction,

initiative, persistence, information use, systems thinking, communication and reflection. However, the

mathematics items in PISA 2021 are not specifically developed according to these skills.

80

Sample Released PISA Items

Sample 1

CHARTS

In January, the new CDs of the bands 4U2Rock and The Kicking Kangaroos were

released. In February, the CDs of the bands No One’s Darling and The Metalfolkies

followed. The following graph shows the sales of the bands’ CDs from January to June.

Translation Note: The term “charts” does not refer to the mathematical term, but to the weekly

listing of the best selling music CDs.

Translation Note: Translate band names with fictitious band names in your language.

Translation Note: The names of the months are shown in abbreviated form in the

graphic. Full names can be used if space allows, as shown in the FRE version.

81

Item 1: CHARTS

PM918Q01

How many CDs did the band The Metalfolkies sell in April?

A 250

B 500

C 1000

D 1270

E

CHARTS SCORING 1

QUESTION INTENT:

Full Credit

Description: Read a bar chart

Mathematical content area: Uncertainty and data

Context: Societal

Process: Interpret

Code 1: B. 500

No Credit


Code 9: Missing.

__________________________________________________________________________

Item 2: CHARTS

PM918Q02

In which month did the band No One’s Darling sell more CDs than the band The Kicking

Kangaroos for the first time?

A No month

B March

C April

D May

CHARTS SCORING 2

QUESTION INTENT:

Description: Read a bar chart and compare the height of two bars


Context: Societal

Process: Interpret

82

Full Credit

Code 1: C. April

No Credit


Code 9: Missing.

Item 3 : CHARTS

PM918Q05

The manager of The Kicking Kangaroos is worried because the number of their CDs that

sold decreased from February to June.

What is the estimate of their sales volume for July if the same negative trend continues?

A 70 CDs

B 370 CDs

C 670 CDs

D 1340 CDs

CHARTS SCORING 5

QUESTION INTENT:

Description: Interpret a bar chart and estimate the number of CDs sold in the

future assuming that the linear trend continues Mathematical content area:

Uncertainty and data

Full Credit

Context: Societal

Process: Employ

Code 1: B. 370 CDs

No Credit


Code 9: Missing.

83

Sample 2

SAILING SHIPS

Ninety-five percent of world trade is moved by

sea, by roughly 50 000 tankers, bulk carriers and

container ships. Most of these ships use diesel

fuel.

© by skysails

Engineers are planning to develop wind power

support for ships. Their proposal is to attach kite

sails to ships and use the wind’s power to help

reduce diesel consumption and the fuel’s impact

on the environment.

Translation Note: “© by skysails”: Do not adapt skysails as this is a registered label.

Item 1: SAILING SHIPS

PM923Q01

One advantage of using a kite sail is that it flies at a height of 150 m. There, the wind speed

is approximately 25% higher than down on the deck of the ship.

At what approximate speed does the wind blow into a kite sail when a wind speed of 24 km/h

is measured on the deck of the ship?

A 6 km/h

B 18 km/h

C C 25 km/h

D 30 km/h

E 49 km/h

Translation Note: In this unit please retain metric units throughout.

SAILING SHIPS SCORING 1

QUESTION INTENT:

Description: Apply calculation of percentage within a given real world situation

Mathematical content area: Quantity

Context: Scientific

Process: Employ

Full Credit

84

Code 1: D. 30 km/h

No Credit


Code 9: Missing.

Item 2: SAILING SHIPS

PM923Q03

Approximately what is the length of the

rope for the kite sail, in order to pull the

ship at an angle of 45° and be at a

vertical height of

150 m, as shown in the diagram

opposite?

A 173 m

B 212 m

C 285 m

D 300 m


Rope

45 º 90 º

150 m

Note: Drawing not to scale.

© by skysails

QUESTION INTENT:

Description: Use Pythagorean Theorem within a real geometric context

Mathematical content area: Space and shape

Context: Scientific

Process: Employ

Full Credit

Code 1: B. 212 m

No Credit


Code 9: Missing.

85

Item 3 : SAILING SHIPS PM923Q04 – 0 1 9

Due to high diesel fuel costs of 0.42 zeds per litre, the owners of the ship NewWave are

thinking about equipping their ship with a kite sail.

It is estimated that a kite sail like this has the potential to reduce the diesel consumption

by about 20% overall.

The cost of equipping the NewWave with a kite sail is 2 500 000 zeds.

After about how many years would the diesel fuel savings cover the cost of the kite sail?

Give calculations to support your answer.

Number of years: .....................................

Translation Note: Change to , instead of . for decimal points, if that is your standard

usage.


QUESTION INTENT:

Description: Solve a real world situation involving cost savings and fuel consumption

Mathematical content area: Change and relationships

Context: Scientific

Process: Formulate

Full Credit

Code 1: A solution from 8 to 9 years is provided with adequate (mathematical) calculations.

86

No Credit

• Diesel consumption per year without a sail: 3.5 million litres, price 0.42 zed/litre,

costs for diesel without a sail 1 470 000 zeds. If 20% is saved with the sail this

results in a saving of 1 470 000 x 0.2 = 294 000 zeds per year. Thus: 2 500 000

/ 294 000 ≈ 8.5, i.e.: After about 8 to 9 years, the sail becomes (financially)

worthwhile.


Code 9: Missing.

Sample 3

87

Item 1 : WHICH CAR?

PM985Q01

Chris wants a car that meets all of these conditions:

• The distance travelled is not higher than 120 000 kilometres.

• It was made in the year 2000 or a later year.

• The advertised price is not higher than 4500 zeds.

Which car meets Chris’s conditions?

A Alpha

B Bolte

C Castel

D Dezal

WHICH CAR? SCORING 1

QUESTION INTENT:

Description: Select a value that meets four numerical conditions/statements set

within a financial context

Full Credit


Context: Personal

Process: Interpret

Code 1: B Bolte.

No Credit


Code 9: Missing.

Item 2: WHICH CAR?

PM985Q02

Which car’s engine capacity is the smallest?

A Alpha

B Bolte

C Castel

D Dezal

88


QUESTION INTENT:

Description: Choose the smallest decimal number in a set of four, in context


Context: Personal

Process: Employ

Full Credit

Code 1: D Dezal.

No Credit


Code 9: Missing.

Item 3 : WHICH CAR? PM985Q03 – 0 1 9

Chris will have to pay an extra 2.5% of the advertised cost of the car as taxes.

How much are the extra taxes for the Alpha?

Extra taxes in zeds: ................................


QUESTION INTENT:

Description: Calculate 2.5% of a value in the thousands within a financial context


Context: Personal Process:

Employ

Full Credit

Code 1: 120.

No Credit


Code 9: Missing.

• 2.5% of 4800 zeds [Needs to be evaluated.]

89

Sample 4

Item 1: GARAGE

PM991Q01

The illustrations below show different “basic” models as viewed from the back. Only one

of these illustrations matches the model above chosen by George.

Which model did George choose? Circle A, B, C or D.

A

B

C

D

GARAGE SCORING 1

90

QUESTION INTENT:

Description: Use space ability to identify a 3D view corresponding to another

given 3D view


Context: Occupational

Process: Interpret

Full Credit

Code 1: C [Graphic C]

No Credit


Code 9: Missing.

Item 2 : GARAGE PM991Q02 – 00 11 12 21 99

91

The roof is made up of two identical rectangular sections.

Calculate the total area of the roof. Show your work.

. ..................................................................................................................................

...................................................................................................................................

...................................................................................................................................

...................................................................................................................................

Translation Note: Decimal parts of measurements shown on diagram will need to be

converted to use a , rather than a . as appropriate.

GARAGE SCORING 2

QUESTION INTENT:

Description: Interpret a plan and calculate the area of a rectangle using the

Pythagorean theorem or measurement


Context: Occupational

Process: Employ

Full Credit

Code 21: Any value from 31 to 33, either showing no working at all or supported by

working that shows the use of the Pythagorean theorem (or including elements

indicating that this method was used). [Units (m 2 ) not required].

• 12√7.25 m 2

• 12 × 2.69 = 32.28 m 2

• 32.4 m 2

Partial Credit

Code 11: Working shows correct use of the Pythagorean theorem but makes a

calculation error or uses incorrect length or does not double roof area.

• 2.5 2 + 1 2 = 6, 12 × √6 = 29.39 [correct use of Pythagoras theorem with

calculation error]

• 2 2 + 1 2 = 5, 2 x 6 x √5 = 26.8 m 2 [incorrect length used]

• 6 × 2.6 = 15.6 [Did not double roof area.]

92

Code 12: Working does not show use of Pythagorean theorem but uses reasonable

value for width of roof (for example, any value from 2.6 to 3) and completes rest

of calculation correctly.

• 2.75 × 12 = 33

• 3 × 6 × 2 = 36

• 12 × 2.6 = 31.2

No Credit


Code 99: Missing.

Sample 5

M309: Building Blocks

• 2.5 × 12 = 30 [Estimate of width of roof lies outside the acceptable range which

is from 2.6 to 3.]

• 3.5 × 6 × 2 = 42 [Estimate of width of roof lies outside the acceptable range

which is from 2.6 to 3.]

Susan likes to build blocks from small cubes like the one shown in the following diagram:

Small cube

Susan has lots of small cubes like this one. She uses glue to join cubes together to make

other blocks.

First, Susan glues eight of the cubes together to make the block shown in Diagram A:

Diagram A

93

Then Susan makes the solid blocks shown in Diagram B and Diagram C below:

Diagram B Diagram C

__________________________________________________________________________

Item 1: BUILDING BLOCKS M309Q01

How many small cubes will Susan need to make the block shown in Diagram B?

Answer: ..................................................cubes.

BUILDING BLOCKS SCORING 1

Full credit

Code 1: 12 cubes.

No credit


Code 9: Missing.

__________________________________________________________________________

Item 2 : BUILDING BLOCKS M309Q02

How many small cubes will Susan need to make the solid block shown in Diagram C?

Answer: ..................................................cubes.

94


Full credit

Code 1: 27 cubes.

No credit


Code 9: Missing.

__________________________________________________________________________


Susan realises that she used more small cubes than she really needed to make a block like

the one shown in Diagram C. She realises that she could have glued small cubes together to

look like Diagram C, but the block could have been hollow on the inside.

What is the minimum number of cubes she needs to make a block that looks like the one

shown in Diagram C, but is hollow?

Answer: ..................................................cubes.


Full credit

Code 1: 26 cubes.

No credit


Code 9: Missing.

__________________________________________________________________________


Now Susan wants to make a block that looks like a solid block that is 6 small cubes long, 5

small cubes wide and 4 small cubes high. She wants to use the smallest number of cubes

possible, by leaving the largest possible hollow space inside the block.

What is the minimum number of cubes Susan will need to make this block?

Answer: ..................................................cubes.

95


Full credit

Code 1: 96 cubes.

No credit


Code 9: Missing.

Sample 6

PISA-like or TIMSS-like:

PISA-like

Domain:

Unit Title:

TAK KUSUT LAGI/NO MORE TANGLES

Scenario :

Proposed stimulus

96

Xstore online advertisement shows two shampoos for sale. Buyer has to pay

RM2.00 for the delivery fee of shampoo A. But shampoo B has no delivery fee.

Lina decides to buy online either one of these shampoos

Iklan online Xstore menunjukkan dua shampu untuk dijual. Pembeli perlu

membayar RM2.00 yuran penghantaran bagi shampu A tetapi shampu B tiada

sebarang yuran dikenakan. Lina bercadang membeli salah satu shampu

tersebut.

ITEM 1: NO MORE TANGLES

Which shampoo is better value for money?

Shampu mana yang lebih nilai untuk wang?

ITEM SPECIFICATION

Question Intent

To compare two costs and select

the less one

Item Code

Content

Quantity

Process

Employ

Cognitive Demand

Average

Context

Personal

Item Format


97

SCORING GUIDE

Full Credit

Code 1 : B

No Credit

Code 0: A

Sample 7

PISA-like

Domain: Relation

Unit Title:

BATTLE OF THE FIELD/PERTEMPURAN PADANG

Scenario :

Proposed stimulus

Dalam satu perlawanan ragbi, Ali bertembung dengan pemain lawan semasa

perebutan bola. Ali merasakan satu tenaga terhasil yang membuatkan dia

98

terhenti berlari.Tenaga itu dikenali sebagai tenaga impuls yang terhasil semasa

pertembungan. Semakin besar nilai F semakin besar impak pertembungan.

In a rugby tournament, Ali collided into an opponent player while struggling for

the ball. He felt a force impacted during the collision that caused him to stop

running. The force is known as the energy of impulse in collisions. The greater

the F, the bigger the impact of collision.

99

Item 1: BATTLE OF THE FIELD/PERTEMPURAN PADANG

Dalam hukum Newton kedua, tenaga impuls ialah

di mana m

ialah berat pemain, t masa pertembungan dalam saat, v ialah kelajuan awal dan

u ialah kelajuan akhir. F ditulis dalam unit newton N.

Kelajuan akhir ialah 0 m/s kerana larian terhenti selepas bertembung.

Berapakah tenaga F yang telah terhasil semasa pertembungan selama 0.5

saat?

In Newton’s Second Law, the force of impulse can be derived from

where m is weight of object, t is time in second, v is initial speed and u is final

speed. F is written in unit of newton, N.

Final speed is 0 m/s because the run stopped after the collision. How much F

energy produced during the collision in 0.5 second?

ITEM SPECIFICATION

Question Intent

To find value from a given formula.

Item Code

Content

Change and relationships

Process

Employ

Cognitive Demand

Average

Context

Scientific

Item Format


100

SCORING GUIDE

Full Credit

Code 1 :

F = 2,100 N

No Credit

Code 9: Missing.


101

Sample 8

PISA-like

Domain:Mathematics Grade 7

Unit Title: Tangram

Scenario:

Proposed stimulus 1

Tangram

A tangram is a square shaped Chinese puzzle consisting of seven geometric

pieces. The pieces can be arranged to make various shapes. Triangles and

quadrilaterals are the basic shapes found in the tangram.

Item 1: TANGRAM

Ahmad accidently dropped the tangram and lost the quadrilateral pieces.

Name the pieces that were lost.

102

ITEM SPECIFICATION

Question intent

Identify types of quadrilaterals in

tangram

Item code

Content Domain

Geometry

Cognitive Process

Knowing

Cognitive Demand

Easy

Item Format


SCORING GUIDE

Code 1: square, parallellogram (both correct) or D and E


Code 9: Missing

Item 2: TANGRAM

Here are four statements about the tangram. Mark X to show whether each

statement is true or false.

Statements True False

The area of F is the same as that of G

The perimeter of B is of the perimeter of the whole

tangram.

The area of D is half the area of A

The total area of D and F is the same as the total

area of C, E and G.

103

ITEM SPECIFICATION

Question intent

Determine area by comparing pieces

Item code

Content Domain

Geometry

Cognitive Process

Knowing

Cognitive Demand

Moderate

Item Format

CMC

SCORING GUIDE

Answer: Yes No Yes Yes in that order

Code 2: Gives 4 correct answers

Code 1: Gives 2 or 3 correct answers

Code 0: Gives 1 or 0 correct answer

Code 9: Missing

104

Item 3: TANGRAM

Ahmad managed to find the missing pieces and rearrange all the pieces to form

Figure 2.

He concluded that the area of Figure 2 is less than the area of Figure 1. Do you

agree? Give your reason.

ITEM SPECIFICATION

Question intent

To undertand that area is conserved even though the

shape is changed

Item code

Content Domain

Geometry

Cognitive Process

Reasoning

Cognitive Demand

Easy

Item Format


105

SCORING GUIDE

Code 1: No. Area is conserved even though the shape is changed


Code 9: Missing

106

The Process

Sample 1

(i) Original Item


PISA

Domain:

Numbers

Unit Title:

CAR TYRE

107

Proposed stimulus

CAR TYRES

Tyres used on the road must have a production date code that displays weeks and year on

the tyre wall. The date code 2704 means that the tyre was made during the 27th week of

2004.

Ramli wants to change his car’s worn tyre. The following table shows the details of several

brands of tyres obtained.

Brand Allo Bettin Concode Danga

Date code 0317 0718 4717 2516

Price (RM) 200 175 150 160

Durability

(km)

Maximum

speed (km/h)

35000 35000 32000 25000

240 220 210 180

Australian Council for Educational Research Limited

19 Prospect Hill Road (Private Bag 55) Camberwell

VIC 3124 Australia t +61 3 9277 5555 f +61 3

9277 5500 www.acer.org

ACN 004 398 145 ABN 19 004 398 145

108

Item 1: CAR TYRE

Ramli wants to buy a tyre that meets the following requirements:

- It was made in Mac 2017 or later

- The price of the tyre must not be more than RM170

- Durability should be at least 30000 km

Which brand of tyre meets Ramli’s preference?

A. Allo

B. Bettin

C. Concode

D. Danga

ITEM SPECIFICATION

Question Intent

Item Code

Content

Process

Cognitive Demand

Context

Item Format

Select a value that meets three

numerical conditions/statements set.

Quantity

Interpret

Medium

Personal

Simple multiple choice

SCORING GUIDE

Full Credit Code 1: C.

No Credit

Code 0: Other responses. Code 9: Missing.

Item 2: CAR TYRE

Which brand of tyre should not exceed 200 km/h speed?

A. Allo

B. Bettin

C. Concode

D. Danga

109

ITEM SPECIFICATION

Question Intent

Item Code

Content

Process

Cognitive Demand

Context

Item Format

Choose the smallest number in a set of four, in context.

Quantity

Employ

Low

Personal

Simple multiple choice

SCORING GUIDE

Full Credit Code 1: D.

No Credit

Code 0: Other responses. Code 9:

Missing.

Item 3 : CAR TYRE

Ramli went to the tyre shop in March 2019. He was told that for every one year lapse from

the date of production, the price of the tyre will reduce by 20% and the durability of the tyre

will drop by 2000 km. Which bran d of tyre is worth buying? Explain your answer.

ITEM SPECIFICATION

Question Intent

Item Code

Content

Process

Cognitive Demand

Context

Item Format

Calculate proportion of price with distance.

Quantity

Interpret and evaluate

High

Personal

Open Constructed Response

110

SCORING GUIDE

Full Credit

Code 1: Concode with suitable reason

The value per kilometre is the lowest

• RM0.0040

• 0.40 sen

• RM120 / 30000 km

No Credit

Code 0: Other responses. Code 9:

Missing.

111

(ii) Reviewed Item

Unit 1: Car Tyre Review

Stimulus:

review of

general and

specific

criteria

Not clear where source of image for tyre and tyre code has come from.

May need to seek permission to use, or have it reproduced by a graphic

artist.

Suggest the following changes to stimulus:

1. change durability of Concode to 25 000 and Danga to 32 000, this

will strengthen MC options for item 1 (as currently written, you can

eliminate option D for two reasons and do not really need to read

further than the first two criteria)

2. change date code for Concode to 4617 as this will help to create a

new question focusing on interpretation of date code in terms of

month and year (separate from the first question which will focus

just on comparing values in table to given criteria)

3. Remove the row titled ‘Maximum speed’ as it is not really used in

any substantial way in the items and the question about the date

code described above could replace the very simple second

question in the unit that attempts to use this row of data.

ACER review

Item ID

Domain

Cognitive

Process/Skill

M001 Mathematics PISA: Interpreting,

applying, and

evaluating

mathematical

outcomes

ACER

Estimated

item

Difficulty

Easy

ACER review

summary:

suggested

action

Implement

suggested

modifications

Item

Entire Item


What is the most important skill in this item? Interpreting a table in light of

given criteria? Or converting weeks of the year to months? It seems like

the interpretation of


ACN 004 398 145 ABN 19 004 398 145

112

Item 2 : Car Tyre MC

ACER review

Item ID

Domain

Cognitive

Process/Skill


applying, and

evaluating

mathematical

outcomes

ACER

Estimated item

Difficulty

Average

ACER review

summary:

suggested

action

Implement

suggested

modifications

Item

Entire Item

Stem


Is very much easier than first question. Probably too easy. Questions

should generally be ordered in a unit from easiest to hardest. Suggest

rewrite this question as one focusing on the date code and conversion

from weeks to months. This new item would remain in the content

category Quantity item

In which month and year was the Concode tyre made?

Year: ___________ Month: ___________

Option A

Option B

Option C

Option D

Scoring guide

ACER review

summary:

suggested

action

113

the table should be the primary focus in this case (i.e. item

content category should be Uncertainty and data). Once

the complication of converting weeks to months is

removed, then the item difficulty would be Easy, but the

skill being test is still very important and authentic in terms

of the context. A separate item could be written about

converting weeks to months.

Stem Change ‘Mac 2017’ to ‘the year 2017’

Option A

Option B

Option C

Option D

Change ‘must not be more than RM 170’ to ‘must be less

than RM 180’ [avoids reading difficulty with ‘not’ and 180

makes it harder to eliminate option ]

Scoring guide

ACER review

summary:

suggested

action

Implement

suggested

modifications


Question intent statement

Suggest modify slightly: ‘Select a value from a given table

that meets three given criteria.’

Question intent statement

matches

the correct response?

If MC, the Key is correct?

If CMC, are the scoring

categories appropriate?

With the above changes as described to make better use

of the full range of three criteria in selecting a tyre, the key

would need to change from C to B

114

Entire Item

Is very much easier than first question. Probably too

easy. Questions should generally be ordered in a unit

from easiest to hardest. Suggest rewrite this question

as one focusing on the date code and conversion from

weeks to months. This new item would remain in the

content category Quantity item

Stem

In which month and year was the Concode tyre made?

Year: ___________ Month: ___________

Option A

Option B

Option C

Option D

Scoring guide

Code 2: Both year and month correct

• Year: 2017 Month: November

• Year: 17 Month: Nov

Code 1: Either year or month correct

• Year: 2017 Month: October

• Year:2007 Month: Nov

• Year: 2017 Month: ______

ACER rev

summary:

suggested

Implemen

modificatio

115


Question intent

statement

Modify to:

‘Identify the month within which a given week in the year will fall’

Question intent

statement

matches

the correct

response?

If MC, the Key

is correct?

If CMC, are the

scoring

categories

appropriate?

Item 3 : Car Tyre CR

ACER review

Item ID

Domain

Cognitive

Process/Skill

M003 Mathematics PISA: Formulating

situations

mathematically

ACER

Estimated item

Difficulty

Difficult

ACER review

summary:

suggested action

Implement

suggested

modifications

Item

Stem


Question as currently written is too open, that is what is ‘worth buying’ is

likely to be interpreted in many different ways depending on different

personal and social values and beliefs. Needs to be made more specific.

Suggest the following reword:

116

‘Ramli went to the tyre shop in the fifth week of 2019. For tyres that were

more than one year old:

• the price was reduced by 20% for every full year that passed since

its production date.

• the durability of the tyre was also reduced by 2000 km for every full

year that passed since its production date. ‘

Question

Scoring guide

Which brand of tyre from the table offered the best durability for its price in

the seventh week of 2019? Show your working.

Full Credit

Code 1: Danga with working showing evidence of

proportional reasoning considering both durability

and cost. Note: working showing an iterative

interpretation of 20% discount per year( i.e. RM

120 reduced to 160 in 2018 and then to RM128 in

2019) OR a cumulative interpretation (ie 2 years

means 40% discount on original price) should

both be accepted as correct, as long as the same

interpretation is applied consistently to all prices

• The value per kilometre is the lowest

• The number of kilometres per RM is highest

• RM0.0034

• 292 km per RM compared to 242 km per

RM for

Allo, 200 km per RM for Bettin and 183km per RM

for

Concode

• RM96 / 28000 km

• 28000km / RM96

ACER review

summary: action

Implement suggested

modifications

No Credit

Code 0: Other responses, including Danga

selected with no working or incorrect working

Danga because it is the oldest tyre so it

gets the biggest discount, making it the

best value for money

Code 9: Missing.

117


Question intent

statement

Description for

each scoring

category

Examples of

student

responses for

each scoring

category

Reason proportionally to solve a multi-step problem involving percentage

discounts and the calculation and comparison of several rates

See above scoring guide

See above scoring guide

(iii) Final Item

PISA-like

Domain:

Nombor

Tajuk unit:

TAYAR KERETA

Stimulus cadangan

TAYAR KERETA

Tayar yang digunakan di jalan raya perlu mempunyai kod tarikh pengeluaran yang

memaparkan minggu dan tahun pada permukaan tayar. Kod tarikh 2704 bermakna tayar

itu dibuat pada minggu ke-27 tahun 2004.

118

Ramli hendak menukar tayar keretanya yang telah haus. Jadual berikut menunjukkan

butiran bagi beberapa jenama tayar.

Jenama Allo Bettin Concode Danga

Kod tarikh 0317 0718 4617 2516

Harga (RM) 200 175 150 160

Ketahanan

(km)

35 000 35 000 25 000 32 000


ACN 004 398 145 ABN 19 004 398 145

Item 1: TAYAR KERETA

Bilakah tayar Concode dibuat berdasarkan bulan dan tahun?

Bulan: _____________ Tahun: _____________

SPESIFIKASI ITEM

Tujuan Soalan

Mengenalpasti bulan di mana minggu dalam

tahun yang diberi berada.

Kod Item

Kandungan

Kuantiti

Proses Mentafsir, mengaplikasi dan menilai hasil

matematik

Aras Kognitif

Mudah

Konteks

Individu

Format item

Respon Konstruktif

PENSKORAN

Kredit Penuh

Kod 2: Kedua-dua bulan dan tahun betul (Terima dalam bentuk angka dan perkataan)

• Bulan : Nov Tahun: 17

• Bulan: November Tahun: 2017

119

Kod 1: Sama ada tahun atau bulan betul

• Bulan: Oktober Tahun: 2017

• Bulan: Nov Tahun: 2007

• Bulan:_______ Tahun: 2017

Tiada Kredit

Kod 0: Lain-lain respon.


Item 3 : TAYAR KERETA

Ramli pergi ke kedai tayar pada minggu kelima tahun 2019. Bagi tayar yang berusia lebih

daripada satu tahun:

• harga dikurangkan sebanyak 20% untuk setiap kali genap setahun dari

tarikh pengeluarannya.

• ketahanan tayar juga berkurang sebanyak 2000 km untuk setiap kali

genap setahun dari tarikh pengeluarannya.

Daripada jadual, jenama tayar yang manakah menawarkan ketahanan terbaik yang

sepadan dengan harganya pada minggu ketujuh tahun 2019? Tunjukkan jalan kerja

anda.

Jenama tayar:

120

SPESIFIKASI ITEM

Tujuan Soalan

Kod Item

Kandungan

Proses

Aras Kognitif

Konteks

Format Item

Menaakul secara berkadaran untuk menyelesaikan

masalah pelbagai langkah yang melibatkan peratus

diskaun dan pengiraan serta perbandingan bagi

beberapa kadar.

Kuantiti

Mentafsir, mengaplikasi dan menilai hasil matematik

S ukar

Individu

Respon Konstruktif

PENSKORAN

Kredit Penuh

Kod 2: Danga, dengan jalan kerja yang menunjukkan penaakulan secara berkadaran

serta mengambil kira kedua-dua faktor ketahanan dan kos.

Nota: Jalan kerja menunjukkan tafsiran berulang bagi diskaun 20% setiap

tahun. (RM160 dikurangkan kepada RM128 pada 2017 dan kemudian kepada

RM102.40 pada 2018) (with or without unit)

• Nilai setiap kilometer yang terendah

• Bilangan kilometer per RM yang tertinggi

• RM0.0037

• 273.44 km per RM untuk Danga berbanding dengan 242.19 km per RM

untuk Allo.

• 235.71 km per RM untuk Bettin dan 191.67 km per RM untuk Concode

• RM102.40 / 28 000 km

• 28 000 km / RM102.40

Kod 1: Tafsiran terkumpul (2 tahun bermaksud diskaun 40% ke atas harga asal)

• Nilai setiap kilometer yang terendah

• Bilangan kilometer per RM yang tertinggi

• RM0.0034

• 292 km per RM untuk Danga berbanding dengan 242 km per RM untuk

Allo, 200 km per RM untuk Bettin dan 183 km per RM untuk Concode

• RM96 / 28 000 km

121

• 28 000 km / RM96

Tiada Kredit

Kod 0: Lain-lain respon, termasuk Danga tanpa jalan kerja, atau jalan kerja yang salah.


• Danga, kerana ia merupakan tayar yang paling lama maka ia

memberikan diskaun tertinggi dan paling menguntungkan.

Sample 2

(i) Original Item


Domain:

Unit Title:

Scenario :

Proposed stimulus

PISA-like

Data

THE VOICE

Michael and John have surveyed four post-paid packages and constructed the following

graph based on their monthly accessing fees and voice call rates. The graph shows the

monthly charges of each packages for call usage ranging from 0 to 360 minutes monthly.

122

190

180

170

160

150

140

130

120

110

100

90

80

70

60

50

40

30

20

10

0

Mobile Package Monthly Charges Comparison

0 60 120 180 240 300 360

Monthly Voice Call (minutes)

Package B

Package C

Package D

Package A

Item 1: THE VOICE


ACN 004 398 145 ABN 19 004 398 145

What does the gradient of the line represent?

ITEM SPECIFICATION

Question Intent

Item Code

Content

Process

Cognitive Demand

Context

Item Format

Interpret meaning of a representation

Relationship and Algebra

Interpret

Medium

Personal

Closed constructed-response

123

SCORING GUIDE

Full Credit

Code 1: Call charges per min ute, or call rate, or RM/minute, or changes of monthly charges

over change s of voice call amount.

No Credit


Code 9: Missing.

Item 2 : THE VOICE

Michael expects to spend more tha n 2 hours but less than 6 hours every month on voice call whereas

John expects less than 2 hours every month on voice call. Suggest the suitable package for each of

them.

ITEM SPECIFICATION

Question Intent

Item Code

Content

Process

Cognitive Demand

Context

Item Format

Interpret selection criteria from the graph and make

decision.


Interpret & Evaluate

Medium

Personal

Closed constructed-response

SCORING GUIDE

Full Credit

Code 1: Package A for Michael, Package C for John.

No Credit

124


Code 9: Missing

Item 3 : THE VOICE

If Michael expects his voice call to exceed 12 hours per month, which package do you suggest for him?

Explain your suggestion.

ITEM SPECIFICATION

Question Intent

Item Code

Content

Process

Cognitive Demand

Context

Item Format

SCORING GUIDE

Interpret information from various choices and

decide the best.


Interpret & Evaluate

High

Personal

Open constructed-response

Full Credit

Code 1:

Package D. [With correct reasoning].

12 hours = 720 minutes.

Clearly, Package B and C will not be less than Package A and D at 720 minutes.

For Package A, the charges increases RM60 for every 360 minutes.

So for 720 minutes usage, the charges for Package A = 120+60 = 180.

For Package D, the charges increases RM40 for every 360 minutes.

So for 720 minutes, the charges for Package D = 130+40 = 170 < Package C.

Hence, Package D is the choice.

No Credit


Code 9: Missing

125

(ii) Reviewed Item

Unit 3: The Voice Review

Stimulus

Unit title Domain Framework ACER review

summary:

suggested action

The Voice

Mathematics PISA Implement suggested

modifications


Stimulus: review of

general and specific

criteria

This stimulus seems a bit dated, since most mobile phone packages

nowadays include unlimited minutes of voice calling, at least to local

numbers. This is why I suggest having the idea of international voice

calls in the stimulus, as these are still generally charged per minute.

Consider the following reword:

‘Michael and John are both looking for a mobile phone plan that

includes international calls.

This graph shows the monthly international voice call charges for

four different mobile phone plans.’

Change horizontal axis label :

‘Total time spent on international voice calls monthly (minutes)’

Item 1: The Voice CR

ACER

review

Item ID

Domain

Cognitive

Process/Skill


applying, and

evaluating

mathematical

outcomes

126

ACER

Estimated item

Difficulty

Average

ACER review

summary:

suggested

action

Implement

suggested

modifications

Item

Stem

Question

Scoring

guide


As suggested above.

As suggested above.

ACER review

summary: action


ACN 004 398 145 ABN 19 004 398 145

Implement

suggested

modifications


Question

intent

statement

Description

for each

scoring

category

Examples of

student

responses

for each

scoring

category

Interpret meaning of the gradient of a linear graph in context

Code 1: Responses that show a correct interpretation of gradient as a rate

of call cost per unit time, or words to that effect.

Include the information from original code descriptor as dot point examples

of student responses under the Code 1 descriptor given above, i.e.

• Call charges per minute

• Call rate

• RM/minute

Suggest leave out last example in original coding descriptor for Code

1 (‘changes of monthly charges over changes of voice call time’) as I

am not convinced this shows an accurate interpretation of the

gradient in relation to the context. Wording seems very artificial

Item 2 : The Voice CR

ACER review

Item ID

Domain

Cognitive

Process/Skill

M008 Data PISA:

Interpreting,

applying, and

127

ACER

Estimated

item Difficulty

Difficult

ACER review

summary:

suggested action

Implement suggested

modifications

evaluating

mathematical

outcomes

Item

Stem


Reword to reduce reading load by using shorter sentences.

‘Michael expects to spend more than 2 but less than 6 hours every month on

international voice calls.

John expects to spend less than 2 hours each month on international voice

calls.’

Question

Scoring guide

Which is the most suitable package for each of them? Explain your answer.

ACER

review

summary:

action

Implement

suggested

modifications


Question

intent

statement

Description

for each

scoring

category

Interpret a set of linear graphs to decide on the most appropriate choices that

fulfil given criteria.

Examples of

student

responses for

each scoring

category

128

Item 3 : The Voice CR

ACER review

Item ID

Domain

Cognitive

Process/Skill

M009 Mathematics PISA:

Formulating

situations

mathematically

ACER

Estimated

item

Difficulty

Difficult

ACER review

summary: suggested

action

Implement suggested

modifications

Item


Stem

Question

It seems odd to give a different expected time for Michael than in previous

question. Suggest make this more general (no actor):

Scoring guide

‘For voice calls exceeding 12 hours per month, which package would be

most suitable?

Explain your answer.’

ACER review

summary:

action

Implement

suggested

modifications


Question

intent

statement

Description

for each

scoring

category

Examples of

student

responses for

each scoring

category

Interpret a set of linear graphs to decide on the graph that would give the

lowest value for a given time, where extrapolation of the graphs is required

Code 1: Package D is selected, with correct reasoning

Include example given as part of one dot point and add a second dot

point showing a graphical approach (extending lines on graph to show

that Package D eventually intersects and dips below the package A line).

129

(iii) Final Item


PISA-like

Domain:

Matematik

Tajuk Unit:

SUARA

Senario:

Stimulus cadangan

Michael dan John sedang mencari pelan telefon bimbit yang juga mempunyai Panggilan

Terus Antarabangsa. Graf menunjukkan caj bulanan bagi empat pakej berbeza berdasarkan

jumlah masa panggilan suara antarabangsa.


Item 1: SUARA

ACN 004 398 145 ABN 19 004 398 145

Apakah yang diwakili oleh kecerunan garis dalam graf tersebut?

130

SPESIFIKASI ITEM

Tujuan Soalan

Mentafsir konteks

maksud kecerunan bagi graf linear dalam

Kod Item

Kandungan

Hubungan dan Algebra

Proses Mentafsir, mengaplikasi,

matematik

dan menilai hasil

Aras Kognitif

Konteks

Format Item

Sederhana

Individu

Respon konstruktif

PENSKORAN

Kredit Penuh

Kod 1 : Respon yang menunjukkan pentafsiran yang betul mengenai kecerunan sebagai

kadar kos panggilan per masa

• Caj panggilan per minit

• Kadar panggilan

• RM / minit

Tiada Kredit

Kod 0: Lain-lain respon

Kod 9: Tiada respon

Item 2: SUARA

Michael menjangkakan penggunaan selama lebih daripada 2 tetapi kurang daripada 6 jam

setiap bulan untuk panggilan suara antarabangsa. John menjangkakan penggunaan kurang

daripada 2 jam setiap bulan untuk panggilan suara antarabangsa. Pakej yang manakah

paling sesuai untuk setiap daripada mereka? Jelaskan jawapan anda.

131

Pakej yang sesuai bagi Michael:

Pakej yang sesuai bagi John:

Terangkan jawapan anda.

SPESIFIKASI ITEM

Tujuan Soalan

Kod Item

Kandungan

Proses

Aras Kognitif

Konteks

Format Item

Mentafsir satu set graf linear untuk menentukan

pilihan paling sesuai yang memenuhi yang kriteria

diberikan


Mentafsir, mengaplikasi, dan menilai hasil

matematik

Sederhana

Individu

Respon konstruktif

PENSKORAN

Kredit Penuh

Kod 1: Pakej yang sesuai bagi Michael: Pakej A

Pakej yang sesuai bagi John: Pakej C

Dengan penerangan yang betul.

Bagi panggilan suara antarabangsa yang lebih daripada 2 jam, garisan Pakej A berada

paling bawah. Oleh itu, Pakej A paling sesuai untuk Michael.

Bagi panggilan suara antarabangsa yang kurang daripada 2 jam, garisan Pakej C

berada paling bawah. Oleh itu, Pakej C paling sesuai untuk John.

Tiada Kredit

132

Kod 0: Lain-lain respon

Kod 9: Tiada respon

Item 3 : SUARA

Pakej yang manakah paling sesuai bagi panggilan suara melebihi 12 jam sebulan?

Pakej:

Jelaskan jawapan anda.

SPESIFIKASI ITEM

Tujuan Soalan

Mentafsir set graf linear bagi menentukan graf yang akan

memberikan nilai paling rendah bagi

masa yang diberikan, di mana graf

perlu

diekstrapolasi / konsep kadar dilibatkan

Kod Item

Kandungan

Proses

Aras Kognitif

Konteks

Format Item


Merumuskan situasi secara matematik

Tinggi

Individu

Respon konstruktif

133

134

THE TEMPLATE

135


Domain:

Unit Title:

Scenario 1:

Proposed stimulus 1

ITEM 1:

ITEM SPECIFICATION

Question Intent

Item Code

Content

Process/Competency

Cognitive Demand

Context

Item Format

136

SCORING GUIDE

Answers

Full Credit

No Credit

137

ITEM CHECKLIST

138

Item Checklist

Criteria Description √ or

X

Stage 1: General criteria

Comments

Are there issues with

copyright permissions to

use material identified as

potential stimulus?

If the source material

cannot be used, can the

material be re-written/redeveloped

using a

similar idea?

Does the stimulus

material allow for linking

items to the cognitive

processes? (PISA)

Does the stimulus

material allow for linking

items to the cognitive

skills? (TIMSS)

Does the stimulus

material allow for

assessing more than

information recall or

isolated factual content

(i.e. does the stimulus

allow for assessing

Higher Order Thinking

skills?)

Does the scope of the

material allow for writing

several items

It may not be possible to use

the source material without

permission of the

author/publisher. Check MOE

guidelines.

Formulating situations

mathematically; or

Employing mathematical

concepts, facts, procedures,

and reasoning; or

Interpreting, applying and

evaluating mathematical

outcomes

Knowing; or

Applying; or

Reasoning

If ‘No’, this material is unlikely

to allow for the assessment

of higher order cognitive

skills.

The items (questions) should

ideally be able to address

several different cognitive

139

(questions) from the

same stimulus material?

Does the stimulus

material allow for more

than one response type?

Does the scope of the

material allow for writing

several items of different

difficulty?

Is the stimulus material

suitable for the

audience?

Is the stimulus for PISA

or for TIMSS?

skills: Does the stimulus

allow for this?

The stimulus should allow for

construction of multiple

choice, complex multiple

choice, restricted response,

and/or open constructed

response items

The stimulus often (but not

always) supports easier and

more difficult items: Does the

stimulus allow for this?

Is the material likely to

engage students in the

targeted year levels? Avoid

materials that look like

‘textbook extracts’. Authentic,

real-life situations to which

students can apply and

transfer their learning are

likely to be more engaging.

Avoid topics that are

controversial or might cause

offense.

PISA employs real-world

texts and contexts as the

assessment has a ‘literacy’

focus, and stimulus material

tends to be ‘rich’ to reflect

this approach. The stimulus

generally allows for 3 or 4

items to be written from it.

TIMSS has a stronger

curriculum focus, with an

emphasis on the usefulness

and application of content as

students engage with the

stimulus material. The

stimulus material usually

allows for 1 or 2 items only.

140

Stage 2: Specific criteria

Is the question stem

aligned to the question

intent?

Is real data used in the

question? If so, is the

source included?

Is the scoring category

appropriate?

Is all of the text,

graphical and/or other

information vital?

Is the text clear?

Is the reading level

(readability)

appropriate?

Question intent should

describe what the student

needs to do to answer the

question.

If data is not real, is a

fictitious name used?

Is the key correct?

Is there a description for each

scoring category?

Are the correct codes used?

Is there unnecessary stimulus

material? The amount of text

to be read or information to

be interpreted should be

minimised, without

compromising the ideas or

situation presented.

Assessing higher order

thinking skills will require

some complexity in the text,

but a balance must be found.

Ensure that the meaning of

the material is clear and

unambiguous. Have

someone else read through

the material and discuss to

ensure the information

presented is not flawed.

Readability refers to how

difficult a text is to be

processed. This is influenced

by attributes of the text such

as word length, vocabulary

complexity, sentence length

and complexity of syntax.

The readability of a text

should be 1 to 2 grade levels

141

Will the wording be clear

in translation?

Stage 3: Higher Order Thinking criteria

below the typically expected

reading level of the

candidates.

Consider translation issues

that might arise, and avoid

using colloquial terms.

Does the material allow

for linking to the specific

outcomes of the TIMSS /

PISA framework,

focusing on higher order

thinking skills?

Are elements of higher

order thinking

predominantly in the

“Applying” and

“Reasoning”?

Does the question allow

the students to

extrapolate from what

they have learned and

apply their knowledge in

new situations?

Is the question aligned

to the correct level of

cognitive demand?

Does the stimulus

material allow for

Mathematics TIMSS

-Knowing (only some

aspects)

-Applying

-Reasoning

Mathematics PISA

-Formulate

-Employ

-Interpret and Evaluate

Applying, focuses on the

ability of students to apply

knowledge and conceptual

understanding to solve

problems or answer

questions. Reasoning, goes

beyond the solution of routine

problems to encompass

unfamiliar situations, complex

contexts, and multistep

problems.

PISA Mathematics Literacy

Cognitive demand-low,

moderate and high

If ‘No’, this material is unlikely

to allow for the assessment

142

assessing more than

information

recall/isolated factual

content?

Is the question context

familiar to students?

of higher order cognitive

skills.

PISA Mathematics Literacy

-societal

-personal

-occupational

-scientific

143

MODUL PEMBINAAN ITEM TIMMS &amp; PISA

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?

MODUL PEMBINAAN ITEM TIMMS & PISA