SEKE 2012 Proceedings - Knowledge Systems Institute

is discussed in detail, including our goals and experimental
design. In Section 5 and 6 the results obtained by a quantitative
and qualitative analysis are presented, respectively. Finally, our
conclusions and future work are given in Section 7.
II. RELATED WORK
According to Offutt [3], one of the three quality criteria on
the dominant Web development is usability. ISO 9241 standard
[13] defines usability as “the extent to which a product can be
used by specified users to achieve specified goals with
effectiveness, efficiency and satisfaction in a specified context
of use”. Although a lot is known concerning the development
of usable Web applications, many of these applications still do
not meet most customers’ usability expectations [8].
Due to the importance of usability, the software
development industry is in vesting in techniques and methods
for the design and evaluation of web applications aiming at
improving the interaction quality [4]. Usability evaluation
methods can be divided into two categories [14]: Usability
Inspections, evaluation methods based on experts’ analyses;
and Usability Tests, in which usability defects are discovered
by observation and interaction with users while they perform
tasks or provide suggestions about the interface design [4].
According to Fernandez et al. [8], most of the published
studies concerning Web usability evaluation are based on user
participation, while inspections methods are being less applied,
even being naturally less expensive, since it does not need any
special equipment [4]. This scenario indicates some research
opportunities on usability inspections techniques, aiming to
reduce the cost of evaluations and encouraging a more frequent
usage of usability evaluation in the industrial environment.
Several usability inspection techniques specific to web
applications have been proposed [6, 9, 15, 16, 17, 18, 19, 20].
Conte et al. [9] proposed the WDP technique (Web Design
Perspective-Based Usability Evaluation Technique). Despite
the feasibility of the WDP technique to detect usability defects
on web applications, novice inspectors had difficulties using
the technique caused by lack of skills such as experience on
usability and inspection, which can affect the outcome of the
inspection [9]. Aiming to guide novice inspectors uncovering
usability problem, Gomes et al. [6] proposed the WDP-RT
(Web Design Perspectives-based Inspection – Reading
Technique). The results of empirical studies to evaluate the
WDP-RT technique indicated that, despite the WDP-RT
technique helping novice inspectors finding usability problems
efficiently and effectively, the inspectors still have difficulty on
applying it [21].
The need to provide an easier usability inspection approach
for novice inspectors motivated our research. Therefore, we
proposed a new usability inspection technique for inspectors
with little knowledge on usability and inspections. Our
technique is called WE-QT (Web Evaluation Question
Technique), and we present it in the next Section.
III. WE-QT (WEB EVALUATION – QUESTION TECHNIQUE)
In order to p rovide an ea sier and effective usability
inspection approach for novice inspectors, we evolve the WDP-
RT technique into another type of inspection technique: the
question based approach, WE-QT [10].
The WE-QT technique consists of a set of questions that
guide inspectors uncovering usability problems. The question
based approach provided by WE-QT simplifies the inspection
process, aiming to r educe the difficulties of the novice
inspectors executing the inspection. Our technique does not
require training on usability, inspection or on the technique
itself before utilizing it. The WE-QT technique hides the Web
perspective concepts used on WDP-RT, as w ell as any other
information that is not being needed at a certain time of the
inspection, aiming not to confuse the inspectors with irrelevant
information. The questions that compose our technique were
developed extracting the main goal from the instructions of
WDP-RT and converting them into questions (Fig. 1).
Figure 1. Mapping: WDP-RT’s instructions into the WE-QT’s questions
The questions are classified into Decision Questions (DQ)
and Sub-Questions (SQ). The DQs are responsible for verifying
the elements existence, while SQs are responsible for the
evaluation of these elements [10]. Item (a) of Fig. 1 illustrates
the mapping from an instruction of the WDP-RT technique into
a DQ of the WE-QT. Once we mapped a DQ, the features that
must be evaluated regarding the element of DQ were extracted
and turned into a SQ (item (b) of Fig. 1). Depending on the
inspector’s answers to the questions, according to the mapping
of WE-QT, the DQs define if the SQs will be displayed.
Therefore, the inspection is customized by the elements of the
application, due to the adaptive flow of the questions. This
process is automated by a support the tool [10]. Table I shows
an extract of the first version of the WE-QT technique.
TABLE I.
EXTRACT OF THE WE-QT TECHNIQUE - FIRST VERSION
Id
Question
Mapping
Yes No
0 Does the page show any messages? 1 10
1 Can you easily see the messages? 2 2
2 Can you easily understand the messages? 3 3
3 Are the messages consistent with the local culture? 4 4
4 Do the messages follow the visual pattern of the application? 5 5
⁞ ⁞ ⁞ ⁞
10
Does the page inform you in which part of the application
you are at?
11 13
11 Can you easily see this information? 12 12
12 Can you easily understand this information? 13 13
13
Are the information and options provided by the page
displayed in a natural and logic order?
14 14
589