2003 IMTA Proceedings - International Military Testing Association

3. Most publishers have chosen not to execute method variance studies for their online tests,
especially from CBT-to-e-testing conversions.
4. For those publishers executing method variance reviews of online products, the findings
have been fairly uniformly one of demonstrable equivalence, even across power and
cognitive tests.
5. Some publishers are presenting a caveat emptor notification to the test administrator prior
to purchase if method variance studies have not been completed on a product formerly
available only in paper/pencil.
Factors Driving Web Assessment
These psychometric and ergonomic considerations aside, the main reason for the advent of web
assessment or e-testing is that test publishers and users perceive cost savings and revenue
enhancement potential with the new format. The new e-testing product offerings are beginning to
have an impact on the test industry similar to what has occurred in recent years in online retailing
within the book-selling industry, delivery of e-learning content in the training industry, and ecruiting
in the recruitment industry.
Web assessment allows for continued test and item refinement by allowing for aggregation of
anonymous raw data that in the past had eluded the test developer, which should make the
development or refinement of test products much faster and less labor-intensive. Further, with
improved document control technologies, publishers can increasingly control their test items
against unlawful copying and copyright violations.
Because e-testing has the potential to attenuate many of the costs traditionally associated with
paper-pencil and desktop-diskette modes of testing, publishers may enjoy better margins without
appreciably raising prices. Some cost-comparison data has demonstrated substantial cost
reductions (Chiocchio, et. al, 2003) and no doubt this area will receive increased scrutiny in
coming years. The now well-established capabilities of instant ordering access, enhanced remote
data harvesting capability, bundled products (i.e., no separate purchases of questionnaires,
answer sheets, scoring templates, shipping, etc), reduced or eliminated scoring and clerical
requirements, and immediately produced results are often posited as valuable improvements for
testing consumers. Thus, web assessment appears to have the promise of an enhanced delivery
solution of value to publishers, test professionals, and ultimately, the test-taking client.
That the test-taking subject should derive benefit from the online paradigm is evidenced by the
rationale of the Canadian Public Service Commission in developing their online pre-employment
testing programs: namely, that the citizenry of Canada would enjoy enhanced access to
employment opportunities by making pre-screening more wide accessible via the web delivery
model (Chiocchio, et. al, 2003). Readers are also encouraged to refer to the proposed Association
of Test Publishers guidelines for web-based e-testing (Harris, 2000) for more information on
evolving standards.
421
45 th Annual Conference of the International Military Testing Association
Pensacola, Florida, 3-6 November 2003