October 2007 Volume 10 Number 4 - Educational Technology ...

It is agreed that the difficulty level ranges from +3, which is very difficult level, to –3, which is a very easy level
(Baker, 2001). Table 1 shows that the level of difficulty of the questions ranges from +2.9 to –2.9. This means that
the questions cover all possible ability levels of students. In table 2, discrimination ranges from –0.3 to –1.22.
Positive discrimination values means that the question is not valid and should be revised. There are questions with
discrimination greater than –0.3. However, we do not include them in the process of selection of items because they
will not be good items because they fall outside the scope of almost all students being tested and would not
contribute anything of value to the results.
Table 2. Analysis Of question difficulty
Max difficulty Min difficulty Average difficulty
2.9 –2.9 0.51
Table 3. Analysis of discrimination
Max discrimination Min discrimination Average discrimination
–0.3 –3.15 –1.22
After the conventional test we tested the adaptive assessment on 24 students. Two of the students did not complete
the exam. According to the hypothesis, we should have modified the formula to shorten the number of questions to
reach the stable level. We started the adaptive test with the conventional level of the students. Each student started
with his previous level and with the consideration that we were certain of 85%, that is, σ = 0.15. The graph in Figure
1 shows the conformance of the adaptive test level and the conventional test level.
4
3
2
1
0
-1
-2
-3
1 3 5 7 9 11 13 15 17 19 21
Coventional Level Adaptive Level
Figure 4. Conformance between conventional and adaptive levels
In Figure 4, about 20% of the points of the conventional levels are not close enough to their corresponding points in
the adaptive level. While the average student level in the conventional tests is –0.17, the average in the adaptive tests
is 0.031. Transforming these values to the 100 scale, we find that these values are 47.14 and 50.51, respectively. We
could say that the averages are almost near each other. The difference between the average of the conventional level
and the adaptive level is –0.20. In the 100 scale it is –3.37.
As an example, the fourth point shows that the conventional level is close to the adaptive level: the conventional
level is about 0.47 and the adaptive level is about 0.2. This means that the difference is 0.27. In the 100 scale, this
difference is about 4 marks. The nineteenth point shows that the conventional level is 0.13 and the adaptive level is
1.9; the difference between these points is 1.8. In the 100 scale, the difference is about 29 marks.
The correlation between the conventional level and the adaptive level is 0.63, which is considered to be highmoderate
relation. The correlation is considered high if it is 0.70 and above.
It is not strange to have some deviation in the results of the two kinds of exams due to human error. After elimination
of 20% of the points that are very far from their corresponding points, we get Figure 5, which is more accurate than
86