July 2006 Volume 9 Number 3 - CiteSeerX

Yin, P.-Y., Chang, K.-C., Hwang, G.-J., Hwang, G.-H., & Chan, Y. (2006). A Particle Swarm Optimization Approach to
Composing Serial Test Sheets for Multiple Assessment Criteria. Educational Technology & Society, 9 (3), 3-15.
A Particle Swarm Optimization Approach to Composing Serial Test Sheets
for Multiple Assessment Criteria
Peng-Yeng Yin and Kuang-Cheng Chang
Department of Information Management, National Chi Nan University, Pu-Li, Nan-Tou, Taiwan 545, R.O.C.
Gwo-Jen Hwang
Department of Information and Learning Technology, National University of Tainan, 33, Sec. 2
Shulin St.,Tainan city 70005, Taiwan, R.O.C.
gjhwang@mail.nutn.edu.tw
Tel: 886-915396558
Fax: 886-6-3017001
Gwo-Haur Hwang
Information Management Department, Ling Tung University, Taichung, Taiwan 40852, R.O.C.
Ying Chan
Graduate Institute of Educational Policy and Leadership, Tamkang University
Tamsui, Taipei County, Taiwan 251, R.O.C.
ABSTRACT
To accurately analyze the problems of students in learning, the composed test sheets must meet multiple
assessment criteria, such as the ratio of relevant concepts to be evaluated, the average discrimination
degree, difficulty degree and estimated testing time. Furthermore, to precisely evaluate the improvement of
student’s learning performance during a period of time, a series of relevant test sheets need to be composed.
In this paper, a particle swarm optimization-based approach is proposed to improve the efficiency of
composing near optimal serial test sheets from very large item banks to meet multiple assessment criteria.
From the experimental results, we conclude that our novel approach is desirable in composing near optimal
serial test sheets from large item banks and hence can support the need of evaluating student learning status.
Keywords
Computer-assisted testing, serial test-sheet composing, particle swarm optimization, computer-assisted
assessment
1. Introduction
As the efficiency and efficacy of the deployment of computer-based tests have been confirmed by many early
studies, many researchers in both technical and educational fields have engaged in the development of
computerized testing systems (Fan et al., 1996; Olsen et al., 1986). Some researchers have even proposed
computerized adaptive testing, which uses prediction methodologies to shorten the length of the test sheets
without sacrificing their precision (Wainer, 1990).
A well-scrutinized test is helpful for teachers wanting to verify whether students well digest relevant knowledge
and skills and for recognition of students’ learning bottlenecks (Hwang et al., 2003a). In a computerized learning
environment, which provides students with greater flexibility during the learning process, information
concerning the student learning status is even more important (Hwang, 2003a). The key to a good test depends
not only on the subjective appropriateness of test items, but also on the way the test sheet is constructed. To
continuously evaluate the learning performance of a student, it is usually more desirable to compose a series of
relevant test sheets to meet a predefined set of assessment criteria such that those test sheets in the same series
will not contain identical test items (or contain only an acceptable percentage of overlapped test items). Because
the number of test items in an item bank is usually large and the number of feasible combinations to form test
sheets thus grows exponentially, an optimal test sheet takes enormous time to build up (Garey & Johnson, 1979).
Previous investigation has even shown that a near-optimal solution is difficult to find when the number of
candidate test items is larger than five thousand (Hwang et al., 2003b), not to mention the composition of a series
ISSN 1436-4522 (online) and 1176-3647 (print). © International Forum of Educational Technology & Society (IFETS). The authors and the forum jointly retain the
copyright of the articles. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies
are not made or distributed for profit or commercial advantage and that copies bear the full citation on the first page. Copyrights for components of this work owned by
others than IFETS must be honoured. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior
specific permission and/or a fee. Request permissions from the editors at kinshuk@ieee.org.
3