27.03.2014 Views

SEKE 2012 Proceedings - Knowledge Systems Institute

SEKE 2012 Proceedings - Knowledge Systems Institute

SEKE 2012 Proceedings - Knowledge Systems Institute

SHOW MORE
SHOW LESS

You also want an ePaper? Increase the reach of your titles

YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.

Conclusion validity: the small number of data points is not<br />

ideal from the statistical point of view, however small sample<br />

sizes are a known problem difficult to overcome.<br />

Construct validity: we measured efficiency and<br />

effectiveness using the same approach proposed in [27]. These<br />

two measures are often used in studies that investigate defect<br />

detection techniques [27], which is also our case.<br />

VII. CONCLUSION AND FURTHER WORK<br />

This paper described an observational study aimed at<br />

eliciting how the usability inspectors apply the WE-QT<br />

technique, a usability inspection technique specifically tailored<br />

for novice inspectors. Both quantitative and qualitative results<br />

of this study provided us with important feedback to improve<br />

the WE-QT technique further.<br />

The quantitative analysis showed that the effectiveness<br />

indicator computed for WE-QT v2 was higher than the<br />

effectiveness measured in the feasibility study. However, other<br />

factors may have influenced this outcome besides the<br />

improvement of WE-QT technique.<br />

Despite the positive result of the quantitative data, the<br />

qualitative analysis showed that novice inspectors still have<br />

difficulties understanding determined questions of the<br />

technique. In addition, the way subjects applied the WE-QT<br />

technique did not correspond to the natural/correct inspection<br />

flow, indicating that the inspection process is not well defined<br />

in the technique. Therefore, according to the adopted<br />

methodology, it is necessary to improve our technique and<br />

execute another observational study.<br />

Future work include: (1) improvement of the technique<br />

based on a detailed analysis of the influence of each<br />

verification in the final list of detected defects; (2) the<br />

replication of the experiment in an industrial environment, and,<br />

at last, (3) further studies comparing the WE-QT technique<br />

with other inspection techniques specific for evaluate Web<br />

applications usability.<br />

ACKNOWLEDGMENTS<br />

The authors would like to acknowledge the support granted by<br />

CAPES process AEX 4982/12-6.<br />

REFERENCES<br />

[1] G. Kappel, B. Proll, W. Retschitzegger, W. Schwinger, "Customization<br />

for ubiquitous web applications: a comparison of approaches",<br />

International Journal of Web Engineering and Technology, v. 1, pp. 79-<br />

111, 2003.<br />

[2] E. Luna, J. Panach, J. Grigera, G. Rossi, O. Pastor, "Incorporating<br />

usability requirements in a test/model-driven web engineering<br />

approach", Journal of Web Engineering, v. 9, n. 2, pp. 132-156, 2010.<br />

[3] J. Offutt, “Quality Attributes of Web Software Applications”, IEEE<br />

Software, v. 19, n. 2, pp. 25-32, 2002.<br />

[4] M. Matera, F. Rizzo, G. T. Carughi, “Web Usability: Principles and<br />

Evaluation Methods”, E. Mendes, N. Mosley, (eds), Web Engineering,<br />

Spinger Verlag, 2006.<br />

[5] E. Mendes, N. Mosley, S. Counsell, “The need for web engineering: an<br />

introduction”, Mendes E., Mosley N. (eds.): ‘Web engineering’<br />

Springer, pp. 1-26, 2006.<br />

[6] M. Gomes, D. Viana, L. Chaves, A. Castro, V. Vaz, et al., "WDP-RT: A<br />

usability inspection reading technique for web applications". VI<br />

Experimental Software Engineering Latin American Workshop, v. 1, pp.<br />

124 - 133, 2009. (In Portuguese).<br />

[7] B. Bonifácio, D. Santos, C. Araújo, S. Vieira, T. Conte, " Applying<br />

Usability Inspection Techniques for Evaluating Web Mobile<br />

Applications". IX Brazilian Symposium of Human Factors on Computer<br />

<strong>Systems</strong>, v. 1., pp. 189-192, 2010. (In Portuguese).<br />

[8] A. Fernandez, E. Insfran, S. Abrahão, "Usability evaluation methods for<br />

the web: A systematic mapping study", Journal Information and<br />

Software Technology, v. 53, n. 8, pp. 789-817, 2011.<br />

[9] T. Conte, J. Massolar, E. Mendes, G. H. Travassos, “Web Usability<br />

Inspection Technique Based on Design Perspectives”, IET Software<br />

Journal, v. 3, n. 2, pp. 106-123, 2009.<br />

[10] P. Fernandes, L. Rivero, B. Bonifácio, D. Santos, T. Conte, " Evaluating<br />

a New Usability Inspection Approach: a Quantitative and Qualitative<br />

Analysis”. VIII Experimental Software Engineering Latin American<br />

Workshop, v. 1, p. 67-76, 2011. (In Portuguese)<br />

[11] F. Shull, J. Carver, G. H. Travassos, “An empirical methodology for<br />

introducing software processes”, ACM SIGSOFT Software Engineering<br />

Notes, v. 26, n. 5, pp. 288-296, 2001.<br />

[12] A. Strauss, J. Corbin, “Basics of Qualitative Research: Techniques and<br />

Procedures for Developing Grounded Theory”, 2ed., SAGE<br />

Publications, 1998.<br />

[13] ISO (1997). ISO 9241-11: Ergonomic requirements for office work with<br />

visual display terminals (VDTs). Part 11 — Guidelines for specifying<br />

and measuring usability. Gènève: International Organisation for<br />

Standardisation.<br />

[14] H. Rocha, M. Baranauskas, “Design and Evaluation of H uman-<br />

Computer Interfaces”, M.C.C., 1. ed. Campinas: Emopi Publisher and<br />

Graphic, v. 1, p. 244, 2003. (in Portuguese).<br />

[15] M. Costabile, M. Matera, "Guidelines for Hypermedia Usability<br />

Inspection", IEEE Computer Society Press, v. 8, n. 1, pp. 66-69, 2001.<br />

[16] L. Triacca, A. Inversini, D. Bolchini, "Evaluating Web usability with<br />

MiLE+". VII IEEE International Symposium on, pp. 22-29, 2005.<br />

[17] M. Blackmon, P. Pols on, M. Kitajima, "Cognitive walkthrough for the<br />

web". Conference on Human Factors in Computing <strong>Systems</strong>, v. 5, pp.<br />

463 – 470, 2002.<br />

[18] L. Filgueiras, S. Martins, C. Tambascia, R. Duarte, "Recoverability<br />

Walkthrough: An Alternative to Evaluate Digital Inclusion Interfaces".<br />

Latin American Web Congress, pp. 71-76, 2009.<br />

[19] M. Allen, L. Currie, S. Bakken, V. Patel, J. Cimino, "Heuristic<br />

evaluation of paper-based Web pages: A simplified inspection usability<br />

methodology", Journal of Biomedical Informatics, v. 39, n. 4, pp. 412 –<br />

423, 2006.<br />

[20] A. Oztekin, A. Nikov, S. Zaim, "UWIS: An Assessment Methodology<br />

For Usability Of Web-based Information <strong>Systems</strong>", Elsevier Science<br />

Inc., v. 82, n. 12, pp. 2038-2050, 2009.<br />

[21] M. Gomes, F. Santos, D. San tos, G. Travassos, T. Conte, “Evolving a<br />

Usability Evaluation Technique through In Vitro and In Vivo Studies”.<br />

IX Brazilian Symposium on Software Quality, v. 1, pp. 229 – 244, 2010.<br />

(In Portuguese)<br />

[22] V. Basili, H. Rombach, “The tame project: towards improvementoriented<br />

software environments”, IEEE Transactions on Software<br />

Engineering, v. 14, n. 6, pp. 758 –773, 1988.<br />

[23] A. Dix, J. E. Finlay, G. D. Abowd, R. Beale, “Human-Computer<br />

Interaction”, 3rd Edition, Prentice-Hall, Inc., 2003.<br />

[24] A. Fernandez, S. Abrah, E. Insfran, "Towards to the validation of a<br />

usability evaluation method for model-driven web development", IV<br />

International Symposium on Empirical Software Engineering and<br />

Measurement, pp. 54, 2010.<br />

[25] J. Nielsen, “Heuristic evaluation”, In: Jakob Nielsen, Mack, R. L. (eds),<br />

Usability inspection methods, Heurisitic Evaluation, John Wiley & Sons,<br />

Inc, 1994.<br />

[26] C. Wöhlin, P. Runeson, M. Höst, M.C. Ohlsson, B. Regnell, A. Wessl,<br />

“Experimentation in software engineering: an introduction”, Kluwer<br />

Academic Publishers, 2000.<br />

[27] C. Denger, R. Kolb, “Testing and Inspecting Reusable Product Line<br />

Components: First Empirical Results”, V International Software Metrics<br />

Symposium, 2006.<br />

593

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!