27.03.2014 Views

SEKE 2012 Proceedings - Knowledge Systems Institute

SEKE 2012 Proceedings - Knowledge Systems Institute

SEKE 2012 Proceedings - Knowledge Systems Institute

SHOW MORE
SHOW LESS

Create successful ePaper yourself

Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.

The Mann-Whitney U test of the qualitative data<br />

(survey responses) revealed that the mean difference in the<br />

responses to all the questions, for all mechanism, were<br />

statistically significant. Table VI shows the mean increases<br />

for all the mechanisms.<br />

We infer from these data that most mechanisms more<br />

than double, and progress feedback almost triples, user<br />

satisfaction. This means that users are on average twice as<br />

satisfied with the applications that provide usability<br />

mechanisms.<br />

TABLE III.RESULTS FOR THE IMPACT OF EACH MECHANISM ON THE<br />

EFFICIENCY ATTRIBUTE.<br />

Mechanism Increase (%)<br />

Global Undo * 46.98<br />

Progress Feedback * 34.15<br />

Structure Text Entry * 62.7<br />

Go Back 14<br />

*Statistically significant.<br />

TABLE IV.RESULTS FOR THE IMPACT OF EACH MECHANISM ON THE<br />

RELATIVE EFFICIENCY ATTRIBUTE.<br />

Mechanism TG relative CG relative Increase (%)<br />

efficiency efficiency<br />

Global Undo 87.3% #<br />

Progres Feed.* 100% 40% 60<br />

Structured TE * 86% 25% 61<br />

Go Back 72% 68% 4<br />

* Statistically significant<br />

# This system implementation is unable to undo an operation unless the mechanism has been<br />

enabled<br />

TABLEV.<br />

Mechanism<br />

RESULTS FOR IMPACT ON EFFECTIVENESS ATTRIBUTE<br />

CG users that<br />

complete the<br />

task<br />

TG users that<br />

complete the<br />

task<br />

Increase<br />

(%)<br />

Globla Undo * 60.9 % 95.7 % 35.7<br />

Progress Feedback 95.7 % 100 % 4.3<br />

Structure Text Entry* 86.96 % 4.4 % 82.56<br />

Go Back * 70 % 18 % 52<br />

TABLE VI.<br />

*Statistically significant.<br />

RESULTS FOR THE IMPACT OF EACH MECHANISM ON THE<br />

SATISFACTION ATTRIBUTE<br />

MECHANISM MEAN INCREASE (%)<br />

Global Undo* 82.28<br />

Progress Feedback* 294.08<br />

Structured Text Entry* 251.50<br />

Go Back* 205.75<br />

*Statistically significant.<br />

III. CONCLUSIONS<br />

The conclusions of this paper refer to the results of the<br />

experiment on the EasyFlight application, as they are much<br />

more reliable than the results for the EasyTheatre<br />

application. The findings from the results analysis were:<br />

• The structured text entry mechanism was<br />

statistically significant in all cases. It also had a<br />

high impact, over 60%, on all usability attributes.<br />

• The progress feedback mechanism did not have a<br />

statistically significant impact on the effectiveness<br />

mechanism. The reason is that users always<br />

manage to complete the tasks, irrespective how<br />

much information they have on the progress of<br />

the task.<br />

• Note that control group users cannot complete the<br />

task if global undo is disabled. On this ground, we<br />

cannot calculate that relative efficiency of this<br />

mechanism.<br />

• The go back mechanism was not statistically<br />

significant for calculating the efficiency and<br />

relative efficiency attributes. A possible reason is<br />

that the browser go back functionality was<br />

enabled. This will be corrected in future<br />

experiments.<br />

We intend to continue testing the EasyFlight tool for<br />

other mechanisms: system status feedback, abort, cancel,<br />

warning and help.<br />

Note that the two software applications used in this<br />

experiment have similar features. Both are e-commerce<br />

transaction systems. Obviously, we have to check the<br />

results of this research on other software application types<br />

in order to generalize the results. To do this, we have now<br />

developed an event-driven home automation application<br />

which we will use to run a similar experiment in the very<br />

near future.<br />

REFERENCES<br />

[1] “ISO/IEC Std. 9126-1: Software engineering – Product quality,”<br />

2001.<br />

[2] “ISO/IEC Std. 9241-11: Ergonomic requirements for office<br />

work with visual display terminals. Part11: "Guidance on<br />

Usability",” 1998.<br />

[3] “ISO/IEC Std. 25010-3: <strong>Systems</strong> and software engineering:<br />

Software product quality and system quality in use<br />

models,” 2009.<br />

[4] Cysneros, L.; Wemeck, V.; Kushniruk, A., “Reusable<br />

<strong>Knowledge</strong> for Satisficing Usability Requirements”.<br />

<strong>Proceedings</strong> of the 2005 13 th IEEE International Conference on<br />

Requirements Engineering.<br />

[5] Juristo, N.; Moreno, A.M.; Sanchez-Segura,M., “Analysing the<br />

impact of usability on Software Design” Journal of <strong>Systems</strong> and<br />

Software, Vol. 80,Issue 9,2007,pp.1506-1516.<br />

[6] Bass, L.; Bonnie, J.; Kates, J., “Achieving Usability Through<br />

Software Architecture”. Technical Report MU/SEI-2001-TR-005<br />

ESC-TR-2001-005, (2001)<br />

[7] Juristo, N.; Moreno, A.M.; Sanchez-Segura M., “Guidelines for<br />

eliciting usability Functionalities”. IEEE Transactions on<br />

Software Engineering, Vol. 33, N°. 11, November 2007, pp.<br />

744-758.<br />

603

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!