Workshops - UbiComp

A Quantitative Evaluation Model of Group User ExperienceZhu Wang, Xingshe Zhou, Haipeng Wang, Hongbo Ni, Ruijuan WuSchool of Computer ScienceNorthwestern Polytechnical UniversityXi’an 710072, Shaanxi, P. R. Chinatransitwang@gmail.comABSTRACTThis paper explores the problem of user experienceevaluation, in particular the quantitative evaluation of groupuser experience, in the ubiquitous computing environments.Firstly, the classification and definition of four differentcategories of user groups are proposed and the notion ofgroup user experience is introduced. Secondly, we analyzethe quantitative evaluation of group user experience fordifferent user groups and establish a uniform evaluationmodel for group user experience. Particularly, we employtwo quantitative metrics, user rating and user attentionduration, as the main criteria for evaluating user experience.At the same time, the intercommunication and differencesamong group members in the capacity of informationacquisition, the degree of correlation with other membersand the weight of impact to the overall group userexperience are taken into account to form a generalquantitative evaluation model of group user experience fordifferent user groups. Finally, we evaluate the effectivenessof the proposed model with preliminary experiments.Author KeywordsUbiquitous computing, evaluation model, group userexperienceACM Classification KeywordsC.4 [Performance of Systems]: Modeling Techniques, H.5.3[Information Interfaces and Presentation]: Group andOrganization Interfaces—Evaluation/methodology, I.6.5[Simulation and Modeling]: Model Development.INTRODUCTIONWith the rapid development of ubiquitous computingtechnology, the evaluation, especially the quantitativeevaluation, of ubiquitous computing systems has become animperative research topic. However, the evaluation ofubiquitous computing systems is extremely difficultbecause of the inherent characteristics of these systems,such as adaptability, heterogeneity and invisibility.Ubiquitous computing takes a user-centered approachthat focuses on providing users with personalized services,such as personalized item recommendation, personalizedinformation presentation, etc., in an unobtrusive way. Itaims at creating more attractive and more personalized userexperience, therefore the evaluation of user experienceCopyright is held by the authors.UbiComp '08 Workshop W2 -- Ubiquitous Systems Evaluation (USE '08)September 21st, 2008This position paper is not an official publication of UbiComp '08.become an important aspect of ubiquitous systemsevaluation. Furthermore, individuals tend to be members ofgroups. For instance, an investigation about the museum,which has been widely adopted as a validation platform forubiquitous computing technologies, showed that only 5%visitors went to the museum all alone, while 45%participated in a guided tour, 30% came with children and20% went to the museum with friends [8]. As a consequent,in order to certify whether ubiquitous computing systemshave enhanced experiences of both individuals and groupseffectively, evaluation models for user experienceespecially group user experience become indispensable.This paper attempts to build a uniform quantitativeevaluation model for group user experience, based on thein-depth analysis of the concept of user experience and theclassification of user groups. We employ two quantitativemetrics, user rating and user attention duration, as the maincriteria for evaluating user experience, and at the same timetake into account both the intercommunication and thediversity among group members (e.g., the receptivity ofinformation, the familiarity with the other members and theimpact extent to the group) to form a general quantitativeevaluation model of group user experience for different usergroups.The rest of this paper is organized as follows: Section2 discusses previous work relevant to this paper. In section3, we present the classification of user groups and thenotion of group user experience. Section 4 describes theevaluation model of group user experience according todifferent user groups and proposes a uniform model ofgroup user experience for different user groups. Preliminaryexperiments are presented in section 5. Finally, Section 6concludes the paper.RELATED WORKA very limited number of research works have beenconducted on the quantitative evaluation of user experienceby so far. Masthoff, J. et al. [5] discussed, based onInteractive TV, the quantitative evaluation of satisfactionexperienced by individuals when sequences of TVprograms are recommended according to different kinds ofalgorithms. This work aimed to model and to predict thesatisfaction experienced by individuals, where satisfactionis treated as an affective state. It proposed modifications tothe algorithms to deal with the effect on an individual’ssatisfaction of that of others in the group. As for thequalitative evaluation of user experience, several39