User Guide for the TIMSS International Database.pdf - TIMSS and ...

C H A P T E R 2 I N S T R U M E N T S
Different types of achievement items were included in the item pools for TIMSS. The
multiple-choice items consisted of a stem and either four or five answer choices. In the
instructions at the front of the test booklets, students were encouraged to choose “the answer
[they] think is best” when they were unsure. The instructions do not suggest or imply that
students should guess if they do not know the answer. In the free-response items, students
were asked to construct their own responses to the test questions by writing or drawing their
answers. These included short-answer items and items where students were asked to provide
extended responses. The free-response items were scored using the two-digit coding system
developed by TIMSS (see Chapter 4).
At Population 1, there were 102 mathematics items, including 79 multiple-choice items, 15
short-answer items, and 8 extended-response items. The science test contained 97 items, of
which 13 were classified as requiring short answers and 10 as requiring more extended
responses. In all, there is a total pool of 235 unique testing minutes in Population 1, 118 for
mathematics and 117 for science.
At Population 2, the overall pool of cognitive items contained 151 mathematics items,
including 125 multiple-choice items, 19 short-answer items, and 7 extended-response items.
There were 135 science items, including 102 multiple-choice items and 33 free-response
items. Population 2 contained a total of 396 unique testing minutes, 198 for mathematics and
198 for science.
2.4 Organization of the Test Booklets
At each population, the test items were allocated to 26 different clusters labeled A through Z.
Also, at each population, the 26 clusters were assembled into eight booklets. Each student
completed one booklet. At Population 1 students were given 64 minutes to complete their
booklets. At Population 2, students were given 90 minutes. The organization of the clusters
and booklets is summarized below, first for Population 1 and then for Population 2. 1
At Population 1, the clusters were either 9 or 10 minutes in length, as shown in Table 2.2. The
core cluster, labeled A, composed of five mathematics and five science multiple-choice items,
was included in all booklets. Focus clusters, labeled B through H, appeared in at least three
booklets, so that the items were answered by a relatively large fraction (three-eighths) of the
student sample in each country. The breadth clusters, largely containing multiple-choice
items, appeared in only one booklet. The breadth clusters are labeled I through M for
mathematics and N through R for science. The free-response clusters were each assigned to
two booklets, so that items statistics of reasonable accuracy would be available. These clusters
are labeled S through V for mathematics and W through Z for science.
1 The organization of the test design is fully documented in Adams and Gonzalez (1996).
2 - 4 T I M S S D A T A B A S E U S E R G U I D E