October 2007 Volume 10 Number 4 - Educational Technology ...

Conclusion
Test 3
Test 4
CBTs
PPTs
CBTs
PPTs
CBTs
NS
ET
NS
ET
NS
ET
NS
ET
NS
ET
2.34 (concept)
2.45 (calculation)
1.49 (concept)
1.46 (calculation)
1.91 (concept)
1.95 (calculation)
1.85 (concept)
1.80 (calculation)
2.24 (concept)
2.20 (calculation)
2.17 (concept)
2.06 (calculation)
1.62 (concept)
1.51 (calculation)
1.37 (concept)
.90 (calculation)
1.68 (concept)
1.40 (calculation)
1.05 (concept)
1.08 (calculation)
.304 .591
.007 .936
.054 .818
.072 .789
.005 .944
.052 .831
.248 .622
3.222 084
1.229 .281
.005 .947
This study demonstrates the feasibility of adopting ET and CBTs to replace conventional NS and PPTs. Under the
same MC testing item construct, the researchers investigate the performance difference among examinees between
partial scoring by the elimination testing and the conventional dichotomous scoring method. This study first builds a
computer-based assessment system, then adopts experimental design to record an examinee’s performance with
different testing tools and scoring approaches. One-way ANOVA does not show sufficient proof to reject the
hypothesis that the performance of students taking CBTs is the same as the performance of students taking PPTs.
This finding is in agreement with Alexander et al. (2001) and Bodman and Robinson (2004). We conclude that the
discrepancy does not exist in the performance of examinees who take PPTs nor in the performance of those who take
such CBTs when ET is used.
Next, the correct number of NS was compared to the number of full knowledge of ET in each test. Data analysis
demonstrates that the number of correct NS items is greater than the full knowledge number of ET for each test,
which shows that ET can distinguish between full knowledge and partial knowledge by partial scoring. The number
of unexpected responses calculated by the Rasch model based on item response theory for dichotomous scoring and
the partial credit model based on graded item response for elimination testing to estimate the examinee ability and
item difficulty parameters reveals that the number of unexpected responses in NS is greater than ET for each test,
which demonstrates that ET reduces the unexpected responses of examinees. ET scoring is helpful whenever
examinees’ partial knowledge and unexpected responses are concerned. Moreover, the instructors can more
accurately assess examinees’ partial knowledge by adopting ET and CBTs, which is not only helpful in teaching, but
also increases examinees’ eagerness and willingness to learn.
Experimental results also indicate that different question content does not influence the performance of examinees on
PPTs or CBTs. This result did not confirm Ager’s study (1993). The first reason might be the course content, an
introductory course focused on principles and concepts of operations management, instead of on lots of complex
calculation. The second explanation is that the subjects in this study are all students from the information
management department. These students are quite used to interacting with computer interfaces; consequently, they
present the difference of performance on PPTs and CBTs is insignificant. We remain aware that the validity of any
experimental study is limited to the scope of the experiment. Since the study involved only two classes and one
107