2003 IMTA Proceedings - International Military Testing Association

they may already have in relation to the different types of R&S features available. In eliciting
views about the Tri-Service website, the researchers also wanted to elicit attitudes towards the
traditional and more novel R&S features that could be accommodated on the Internet.
However, a large element of this latter research activity was aimed at canvassing opinion
towards specific interface design elements, in order that, for example, the physical design of
the site could be improved with respect to ease of use, preferences for colours, removal of
constraints etc.
The methodology used to address these objectives is described, and selected results follow. It
is beyond the remit of this paper to give a full account of results specific to the interface
design. Rather, results are restricted to applicant access and their interest and willingness to
use different e-selection features.
METHODOLOGY
Three research activities were designed to address the research issues.
Internet and Careers Survey (ICS): A survey design was used whereby a questionnaire was
developed and administered to students at secondary schools around the UK. The survey was
designed in consideration of available literature and recent public statistics about Internet
access. In addition to access, the questionnaire also included items related to frequency of
Internet use, preferred media, and career search activities. A team of researchers visited the
schools to administer the questionnaires to classes of students aged between 14 and 18 years.
Five schools participated in the study, and completed questionnaires were obtained from 170
students.
Web site usability trials (UTs) – structured walkthroughs: A semi-structured evaluation
protocol was designed in consideration of usability guidelines and a working definition of
usability (Booth, 1989). Booth separates concepts of usability into four key areas:
1. Effectiveness (or Ease of Use): This concerns how easy the site is to use. In simplistic
terms, it relates to the ease with which the system can be used e.g. to get from A to B, and the
effectiveness of the design to allow people to use it.
2. Learnability: This relates to the degree to which the design used e.g. on one website
supports the use of knowledge acquired from previous exposure to other systems. For
example, users should be able to transfer their knowledge of other sites to learn how to use a
new site quickly. Additionally, the site should be flexible enough to support previous
experience, e.g. such that if someone wants to use keyboard shortcuts, this should be
supported.
3. Attitude (Likeability): This covers a broader spectrum. It relates to issues of personal taste
e.g. colour, factors which increase frustration, anxiety, or pleasure and any other subjective
feeling resulting from interaction with the system.
4. Usefulness (or Functionality): This relates to the degree to which the user can and wants to
use the system (or its applications) to achieve their goals. That is, it is not about how easy the
system is to use, it is about the worth of what the web site offers.
Four features of the web site were explored in detail. These included: an application form; a
biodata questionnaire; a medical questionnaire; and an eligibility form. Students were
randomly allocated to one of the four features.
441
45 th Annual Conference of the International Military Testing Association
Pensacola, Florida, 3-6 November 2003