User Guide for the TIMSS International Database.pdf - TIMSS and ...

D A T A B A S E F I L E S C H A P T E R 7
Reliability Variable Names
The variable names for the Original Code, Second Code, and Agreement Code variables are
based on the same general naming system as that for the Original Code variables shown in Table
7.6. The three reliability variables may be differentiated by the second character in the variable
name:
· Original Code Variable: Second Character = S (e.g. ASSSR01, BSPM42)
· Second Code Variable: Second Character = R (e.g. ARSSR01, BRPM42)
· Agreement Code Variable: Second Character = I (e.g. AISSR01, BIPM42)
Reliability Variable Code Values
The values contained in both the Original Code and Second Code variables are the two-digit
diagnostic codes obtained using the TIMSS scoring rubrics. The Agreement Code variable has
three different values depending on the degree of agreement between the two coders:
0 = Identical codes (both digits in the diagnostic codes are identical)
1 = Identical score but different diagnostic code (first digits are the same; second digits
are different)
2 = Different score (both the first and second digits are different)
In general, the response code data contained in the Original Code Variables are identical to those
contained in the Student Written Assessment or Student Performance Assessment files. In some
cases, however, the response codes for specific items were recoded after a review of the
international item statistics revealed inconsistencies in the original coding guides or showed that
the original codes were not functioning as desired (see Martin and Mullis, 1996). The recoded
diagnostic code values were used in computing the achievement scores reflected in the
international reports. Table 7.11 lists the recodings made to the Population 1 and Population 2
written assessment and performance assessment items. These recodes are reflected in the Written
Assessment and Performance Assessment data files.
For the items indicated in Table 7.11, the response codes in the Student Written Assessment or
Student Performance Assessment files reflect the recoded values. In contrast, the Original Code
Variables in the coding reliability files contain the original unrecoded response codes. This was
done so that the coding reliability measure indicated in the Agreement Code Variables was based
on the original coding guides used during the free-response coding sessions conducted in each
country. One exception to this is that any nationally defined diagnostic codes employed in
individual countries (*7 or *8) were recoded to the "other" category (*9) within the same
correctness level prior to the computation of the Code Agreement Variables.
T I M S S D A T A B A S E U S E R G U I D E 7 - 25