SEKE 2012 Proceedings - Knowledge Systems Institute
SEKE 2012 Proceedings - Knowledge Systems Institute
SEKE 2012 Proceedings - Knowledge Systems Institute
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
Conclusion validity: the small number of data points is not<br />
ideal from the statistical point of view, however small sample<br />
sizes are a known problem difficult to overcome.<br />
Construct validity: we measured efficiency and<br />
effectiveness using the same approach proposed in [27]. These<br />
two measures are often used in studies that investigate defect<br />
detection techniques [27], which is also our case.<br />
VII. CONCLUSION AND FURTHER WORK<br />
This paper described an observational study aimed at<br />
eliciting how the usability inspectors apply the WE-QT<br />
technique, a usability inspection technique specifically tailored<br />
for novice inspectors. Both quantitative and qualitative results<br />
of this study provided us with important feedback to improve<br />
the WE-QT technique further.<br />
The quantitative analysis showed that the effectiveness<br />
indicator computed for WE-QT v2 was higher than the<br />
effectiveness measured in the feasibility study. However, other<br />
factors may have influenced this outcome besides the<br />
improvement of WE-QT technique.<br />
Despite the positive result of the quantitative data, the<br />
qualitative analysis showed that novice inspectors still have<br />
difficulties understanding determined questions of the<br />
technique. In addition, the way subjects applied the WE-QT<br />
technique did not correspond to the natural/correct inspection<br />
flow, indicating that the inspection process is not well defined<br />
in the technique. Therefore, according to the adopted<br />
methodology, it is necessary to improve our technique and<br />
execute another observational study.<br />
Future work include: (1) improvement of the technique<br />
based on a detailed analysis of the influence of each<br />
verification in the final list of detected defects; (2) the<br />
replication of the experiment in an industrial environment, and,<br />
at last, (3) further studies comparing the WE-QT technique<br />
with other inspection techniques specific for evaluate Web<br />
applications usability.<br />
ACKNOWLEDGMENTS<br />
The authors would like to acknowledge the support granted by<br />
CAPES process AEX 4982/12-6.<br />
REFERENCES<br />
[1] G. Kappel, B. Proll, W. Retschitzegger, W. Schwinger, "Customization<br />
for ubiquitous web applications: a comparison of approaches",<br />
International Journal of Web Engineering and Technology, v. 1, pp. 79-<br />
111, 2003.<br />
[2] E. Luna, J. Panach, J. Grigera, G. Rossi, O. Pastor, "Incorporating<br />
usability requirements in a test/model-driven web engineering<br />
approach", Journal of Web Engineering, v. 9, n. 2, pp. 132-156, 2010.<br />
[3] J. Offutt, “Quality Attributes of Web Software Applications”, IEEE<br />
Software, v. 19, n. 2, pp. 25-32, 2002.<br />
[4] M. Matera, F. Rizzo, G. T. Carughi, “Web Usability: Principles and<br />
Evaluation Methods”, E. Mendes, N. Mosley, (eds), Web Engineering,<br />
Spinger Verlag, 2006.<br />
[5] E. Mendes, N. Mosley, S. Counsell, “The need for web engineering: an<br />
introduction”, Mendes E., Mosley N. (eds.): ‘Web engineering’<br />
Springer, pp. 1-26, 2006.<br />
[6] M. Gomes, D. Viana, L. Chaves, A. Castro, V. Vaz, et al., "WDP-RT: A<br />
usability inspection reading technique for web applications". VI<br />
Experimental Software Engineering Latin American Workshop, v. 1, pp.<br />
124 - 133, 2009. (In Portuguese).<br />
[7] B. Bonifácio, D. Santos, C. Araújo, S. Vieira, T. Conte, " Applying<br />
Usability Inspection Techniques for Evaluating Web Mobile<br />
Applications". IX Brazilian Symposium of Human Factors on Computer<br />
<strong>Systems</strong>, v. 1., pp. 189-192, 2010. (In Portuguese).<br />
[8] A. Fernandez, E. Insfran, S. Abrahão, "Usability evaluation methods for<br />
the web: A systematic mapping study", Journal Information and<br />
Software Technology, v. 53, n. 8, pp. 789-817, 2011.<br />
[9] T. Conte, J. Massolar, E. Mendes, G. H. Travassos, “Web Usability<br />
Inspection Technique Based on Design Perspectives”, IET Software<br />
Journal, v. 3, n. 2, pp. 106-123, 2009.<br />
[10] P. Fernandes, L. Rivero, B. Bonifácio, D. Santos, T. Conte, " Evaluating<br />
a New Usability Inspection Approach: a Quantitative and Qualitative<br />
Analysis”. VIII Experimental Software Engineering Latin American<br />
Workshop, v. 1, p. 67-76, 2011. (In Portuguese)<br />
[11] F. Shull, J. Carver, G. H. Travassos, “An empirical methodology for<br />
introducing software processes”, ACM SIGSOFT Software Engineering<br />
Notes, v. 26, n. 5, pp. 288-296, 2001.<br />
[12] A. Strauss, J. Corbin, “Basics of Qualitative Research: Techniques and<br />
Procedures for Developing Grounded Theory”, 2ed., SAGE<br />
Publications, 1998.<br />
[13] ISO (1997). ISO 9241-11: Ergonomic requirements for office work with<br />
visual display terminals (VDTs). Part 11 — Guidelines for specifying<br />
and measuring usability. Gènève: International Organisation for<br />
Standardisation.<br />
[14] H. Rocha, M. Baranauskas, “Design and Evaluation of H uman-<br />
Computer Interfaces”, M.C.C., 1. ed. Campinas: Emopi Publisher and<br />
Graphic, v. 1, p. 244, 2003. (in Portuguese).<br />
[15] M. Costabile, M. Matera, "Guidelines for Hypermedia Usability<br />
Inspection", IEEE Computer Society Press, v. 8, n. 1, pp. 66-69, 2001.<br />
[16] L. Triacca, A. Inversini, D. Bolchini, "Evaluating Web usability with<br />
MiLE+". VII IEEE International Symposium on, pp. 22-29, 2005.<br />
[17] M. Blackmon, P. Pols on, M. Kitajima, "Cognitive walkthrough for the<br />
web". Conference on Human Factors in Computing <strong>Systems</strong>, v. 5, pp.<br />
463 – 470, 2002.<br />
[18] L. Filgueiras, S. Martins, C. Tambascia, R. Duarte, "Recoverability<br />
Walkthrough: An Alternative to Evaluate Digital Inclusion Interfaces".<br />
Latin American Web Congress, pp. 71-76, 2009.<br />
[19] M. Allen, L. Currie, S. Bakken, V. Patel, J. Cimino, "Heuristic<br />
evaluation of paper-based Web pages: A simplified inspection usability<br />
methodology", Journal of Biomedical Informatics, v. 39, n. 4, pp. 412 –<br />
423, 2006.<br />
[20] A. Oztekin, A. Nikov, S. Zaim, "UWIS: An Assessment Methodology<br />
For Usability Of Web-based Information <strong>Systems</strong>", Elsevier Science<br />
Inc., v. 82, n. 12, pp. 2038-2050, 2009.<br />
[21] M. Gomes, F. Santos, D. San tos, G. Travassos, T. Conte, “Evolving a<br />
Usability Evaluation Technique through In Vitro and In Vivo Studies”.<br />
IX Brazilian Symposium on Software Quality, v. 1, pp. 229 – 244, 2010.<br />
(In Portuguese)<br />
[22] V. Basili, H. Rombach, “The tame project: towards improvementoriented<br />
software environments”, IEEE Transactions on Software<br />
Engineering, v. 14, n. 6, pp. 758 –773, 1988.<br />
[23] A. Dix, J. E. Finlay, G. D. Abowd, R. Beale, “Human-Computer<br />
Interaction”, 3rd Edition, Prentice-Hall, Inc., 2003.<br />
[24] A. Fernandez, S. Abrah, E. Insfran, "Towards to the validation of a<br />
usability evaluation method for model-driven web development", IV<br />
International Symposium on Empirical Software Engineering and<br />
Measurement, pp. 54, 2010.<br />
[25] J. Nielsen, “Heuristic evaluation”, In: Jakob Nielsen, Mack, R. L. (eds),<br />
Usability inspection methods, Heurisitic Evaluation, John Wiley & Sons,<br />
Inc, 1994.<br />
[26] C. Wöhlin, P. Runeson, M. Höst, M.C. Ohlsson, B. Regnell, A. Wessl,<br />
“Experimentation in software engineering: an introduction”, Kluwer<br />
Academic Publishers, 2000.<br />
[27] C. Denger, R. Kolb, “Testing and Inspecting Reusable Product Line<br />
Components: First Empirical Results”, V International Software Metrics<br />
Symposium, 2006.<br />
593