October 2007 Volume 10 Number 4 - Educational Technology ...

CBTs shorten testing time, give instantaneous results, and increase test security. The online testing format of an
exam can be configured to enable instantaneous feedback to the student, and can be more easily scheduled and
administered than PPTs (Gretes & Green, 2000; Bugbee, 1996).
The test requires no paper, eliminating the battle for the copy machine. The option of printing the test is always
available.
The student can take the test when and where appropriate. Even taking the test at home is an option if the
student has Internet access at home. Students can wait until they think they have mastered the material before
being tested on it.
The test is a valuable teaching tool. CBTs provide immediate feedback, requiring the student to get the correct
answer before moving on.
CBTs save 100% of the time taken to distribute the test to test-takers (a CBT never has to be handed out).
CBTs save 100% of the time taken to create different versions of the same test (re-sequencing questions to
prevent an examinee from cheating by looking at the test of the person next to him).
However, CBTs have some disadvantages:
The test format is normally limited to true/false and MC. Computer-based automatic grading cannot easily
judge the accuracy of constructed-response questions such as short-answer, problem-solving exercises, and
essay questions.
When holding an onsite test, instructors must prepare many computers for examinees, and be prepared for the
difficulties caused by computer crashes.
The computer display is not suitable for question items composed of numerous words, since the resolution
might make the text difficult to read (Mazzeo & Harvey, 1988).
Most items in mathematics and chemistry testing need manual calculation. The need to write down and
calculate answers on draft paper might decrease the answering speed (Ager, 1993).
Research method
This study first constructs a computer-based assessment system, and then adopts experimental design to record an
examinee’s performance with different testing tools and scoring approaches. This section describes research
hypotheses, variables, the experimental design, and data collection.
System configuration
As well as providing a platform for computer-based ET, this system implements the Rasch one-parameter logistics
item characteristics curve (ICC) model for dichotomous scoring and the grade-response model for partial scoring of
ET to estimate the item and ability parameters (Hambleton & Swaminathan, 1985; Wright & Masters, 1982; Wright
& Stone, 1979; Zhu, 1996; Zhu & Cole, 1996). Furthermore, item difficulty analysis allows the system to maintain a
high-quality test bank. shows the system configuration, which comprises databases and three main subsystems: (1)
The computer-based assessment system is the platform used by examinees to take tests, and enables learning
progress to be tracked, grades to be queried, and examinees’ response patterns to be recorded in detail; (2) the
testbank management system, the platform used by instructors to manage testing items and examinee accounts; (3)
the test diagnosis system, which collects data from the answer record and the gradebook database to analyze the
difficulty of test items and the ability of examinees. These subsystems are described in detail as follows (Figure 1):
Computer-based assessment system
The computer-based assessment system links to the answer record, and the gradebook database collects the complete
answer record and incorporates a feedback mechanism that enables examinees to check their own learning progress
and increase their learning efficiency by means of constructive interaction. The main functions of the computerbased
assessment system are as follows: The system first verifies the examinee’s eligibility for the test and then
displays the test information, including allowed answering time, scoring methods, and test items. The system permits
examinees to write down keywords or mark on the test sheet, as in a paper test. The complete answering processes
99