Visualization of Diversity in Large Multivariate Data Sets

More documents

Recommendations

Info

1060 IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL. 16, NO. 6, NOVEMBER/DECEMBER 2010overall diversity looks like under GH is much more ambiguous (evenly“spread out” glyphs with evenly distributed colors, shapes, etc.; seeFig. 3(c)). On the other hand, using DM, it was likely much easier forparticipants to understand exactly how very low and very high diversityappear visually (very low and very high total color density of theentire visual region, respectively; see Figs. 1 (a) and (c)), and we believethis led them to be more confident in choosing responses at bothends of the diversity spectrum when using DM for Q2 questions.Analysis of Results for Q3. What is the most/least diverse attributein the data set represented in this picture? The results for Q3 verymuch favored DM. There was convincing evidence for an effect of visualizationmethod on error distance for both collections of data sets,A and B, F(1,38)=75.54, p = 1.45 × 10 −10 and F(1,38)=13.565,p = 0.0007, respectively. In addition, there was suggestive but inconclusiveevidence for an effect of visualization method on response timein phase 1, F(1,38)=3.50, p = 0.07. These results appear to confirmour initial hypothesis that users would perform better—both in termsof error distance and response time—when making judgments aboutthe diversity of a single attribute when using DM than when using GH.Interestingly, participants using GH appeared to perform worse onQ3 questions where the correct answer was an attribute assigned to aspatial axis, likely due to GH’s ambiguous many-to-one spatial mapping.In contrast, participants using DM did not appear to favor anysingle attribute for questions of this type. Again, this suggests thatDM’s treatment of all attributes as equal is one of its strengths.Analysis of Results for Q4. Which value of attribute X contains themost/least objects? As with Q3, the results for Q4 very much favoredDM. For questions of this type, there was convincing evidence for aneffect of visualization method on error distance for both collections ofdata sets A and B, F(1,38)=7.58, p = 0.009 and F(1,38)=25.18,p = 1.26 × 10 −5 , respectively, and there was suggestive but inconclusiveevidence for an effect of visualization method on response time inphase 1, F(1,38) =2.61, p = 0.11. Again, these results support ourinitial hypothesis that users would be able to more quickly and moreaccurately make judgments about relative abundances within a singleattribute when using DM than when using GH.Summary. The results across Q1–Q4 consistently supported ourhypothesis that users would be able to make more accurate judgmentsabout various aspects of the diversity of data when using DM thanwhen using GH. While we found some evidence suggesting that usersperformed more quickly with DM than with GH, these results werenot conclusive. Similarly, we found no conclusive evidence that sizeof data set had an effect on user performance for questions of type Q1.6.3 Subjective EvaluationAfter each participant answered all of the questions of types Q1–Q4for a particular method, he or she also completed a short questionnaireon that method. The questionnaire, whose form we adopted from[33], consisted of nine Likert-style statements, where participants wereasked to indicate their level of agreement on a scale of 1 (strongly disagree)to 5 (strongly agree), and three open-ended questions.Table 4 lists each of the Likert-style questions along with the participants’mean responses for both GH and DM. Participants slightlyfavored DM over GH in making judgments of diversity componentsand this is consistent with their performance in the objective portionof the study. Participants also slightly favored DM over GH in termsof applicability, ease of understanding, and affinity.In addition to the Likert-style statements, the questionnaires includedthe following three open-ended questions:O1) What aspect(s) of this method did you like most?O2) What aspect(s) of this method did you dislike most?O3) If possible, how would you change this method to improve it?Many participants indicated an affinity for GH because it was intuitive,in that, as the diversity of the underlying data increased, sotoo did the diversity of the visual properties (color, shape, size, etc.)of the generated visualization. On the other hand, many participantsexpressed concern about GH’s ambiguous spatial layout, which theyfound confusing.Table 4. Mean responses to each of nine Likert-style statementspresented to participants immediately after using each visualizationmethod. These responses are based on a scale of 1 (strongly disagree)to 5 (strongly agree). Standard deviations are shown in parentheses.Statement GH DML1) I was able to compare the diversity of two data 3.75 (0.81) 3.93 (0.92)sets using this method.L2) I was able to judge the diversity of a single 3.63 (0.90) 4.25 (0.84)data set using this method.L3) I was able to determine the most/least diverse 3.58 (0.96) 4.15 (0.86)attributes in a data set using this method.L4) I was able to determine the ethnicity with the 4.05 (0.88) 4.28 (0.82)most/least objects using this method.L5) After the initial training session, I knew how 3.33 (0.83) 3.55 (0.99)to use this method well.L6) After answering all of the questions, I knew 3.74 (0.88) 3.88 (0.91)how to use this method well.L7) There are definitely times that I would like to 3.20 (1.04) 3.75 (0.93)use this method.L8) I found this method to be confusing. 3.38 (1.21) 2.77 (1.13)L9) I liked using this method. 2.95 (0.96) 3.50 (1.01)Participants indicated that they liked the “clean layout” of DM; thesimplicity of comparing color opacity under DM; and its ability toeasily handle different data set sizes. On the other hand, some participantsdisliked comparing the diversity of an attribute with severalbuckets (e.g. ethnicity) to that of an attribute with only a few buckets(e.g. gender). Interestingly, though this appears to be an issue with GHas well, participants did not seem to notice it when using GH.Finally, most participants (29 out of 40) preferred DM to GH. Ingeneral, participants tended to feel GH would be best suited for judgingthe overall diversity of a data set, especially to determine if the setis not diverse. Interestingly, this is in direct contradiction to their performancein questions Q1 and Q2 which favored DM. In contrast, participantsgenerally believed DM would be useful for investigating thedata more deeply and examining the diversity of individual attributes.7 DISCUSSION AND FUTURE WORKWe have presented 1) an infrastructure for studying the problem of diversityvisualization and 2) a novel representation for visualizing thediversity of a large set of multivariate objects. The infrastructure includesa precise definition of diversity that takes both richness andevenness into account, a method for generating synthetic data of controllablelevels of diversity, and a formal study design for evaluatingdiversity visualization representations. Based on this definition andstudy design, we developed and evaluated our approach to diversityvisualization, the Diversity Map, which is based loosely on ideas fromboth parallel coordinates and small multiple histograms. We show thatthe Diversity Map allows users to consistently and as or more accuratelyjudge elements of diversity than the only other existing methoddesigned to visualize diversity. While we believe we have taken apositive step in understanding diversity visualization, there are severalissues left to address.Study Design Issues. First, while our study design focuses on staticvisualizations only, both DM and GH are interactive visualizations.We avoided interactive features to limit the scope of our study to firstunderstand the merits and shortcomings of DM and GH as representations.Future work will address the interactive capabilities of DM.Additionally, implementing GH required us to choose a mappingof attributes to the various visual properties of the representation (thethree spatial axes, color, size, shape, etc.). While we based our mappingon the one used by Pearlman et al. [25], our choices here nonethelessrepresent a possible threat to construct validity.Finally, our study does not include a specific question to determinethe richness of variety of an attribute. At first glance, it would appearthat richness of variety was obvious in both methods. However,while richness is clearly communicated in DM and in the non-spatialattributes of GH (e.g. color, shape, size), it is not clear how well richnessis communicated in the spatial axes of GH (e.g. the richness of
PHAM ET AL: VISUALIZATION OF DIVERSITY IN LARGE MULTIVARIATE DATA SETS1061SAT scores in Figure 3 is ambiguous). The evaluation study wouldbenefit from explicit attention to the ability to communicate richness.Limitations of DM. The Diversity Map representation (DM) itselfis also not without limitations. First, DM is currently designed to visualizeonly categorical data, requiring a discretization of quantitativeattributes. Second, the static visualization provides limited insight intothe relationships between attributes. However, variations on the interactiveversion of DM can address these problems. For example, traditionalparallel coordinates poly-lines can be selectively displayed overDM to allow the user to view the actual quantitative attribute values.These poly-lines also allow the user to see and select individual objects,which are currently not visible in the static DM visualization aspresented. Filtering techniques are also implemented in the interactiveversion of DM to allow users to perform queries. For example, theuser can constrain a single attribute to one or more particular values(buckets) using the mouse. The remaining attributes then display thediversity of only those objects that fall within the specified range of thefiltered attribute. With filtering, users can answer questions regardingthe relationship between two attributes such as “In what bucket in attributeX are objects most/least diverse in attribute Y?”.In future work, we will explore other interaction features includinguser-defined orderings for nominal-valued attributes, user-definedordering of the attribute axes themselves, and user specification of attributevalue ranges (over multiple buckets) of interest for interactivefiltering. In addition, we plan to investigate mechanisms for constructingsets of objects of a desired diversity. Finally, we also plan to explorethe advantages of small multiples of DMs for trend analysis.While the Diversity Map representation scales well with the numberof objects to be visualized, like many multivariate visualization methods,scaling with an increase in attributes is limited by screen space.Likewise, the number of buckets for any one attribute is also similarlylimited, and it is not clear how “small” a bucket can be made beforethe representation becomes ineffective. Studies to understand theselimitations are left for future work.The Diversity Map representation, like many others, requires initialtraining for users to be effective in reading the visualization. Indeed,many pilot users of the visualization found the representation counterintuitive.They assumed that if a representation is to convey diversity,high diversity should be shown with an image in which all of the objectslook different, however, in our implementation, high diversity resultsin a uniform image (see Fig. 1(c)). This confusion stems from theusers associating each box with an individual object to be visualized.Once users understood that DM did not display individual objects, butrather their distribution over the attribute space, they were much morereceptive and able to interpret the visualizations consistently as shownin the study. We believe that this representation, where the space fillingeffect denotes diversity, helps the user establish a baseline for highdiversity. In fact, this is supported by the results of our formal study.While participants tended to underestimate diversity when using theGH method, where more diversity implies more dissimilar symbols,they were able to more accurately assign absolute diversity values aswell as compare diversity between two data sets using DM.Note that we empirically chose white as the background color andred as the foreground color in DM. However, we have since foundstudies indicating that blue may be a more appropriate foregroundcolor, since our eyes are known to be more sensitive to changes inblue than in red [21]. Additionally, we empirically used the squareroot-based normalization in determining the color opacity (α-value)of buckets to help make buckets corresponding to attribute values withlow abundance more recognizable. Nevertheless, this ad-hoc scalingfactor is not necessarily a preferred choice by all users. In fact, ecologistsmay prefer a log transformation to accommodate species whoseabundances span multiple orders of magnitude [23]. Moreover, wecould employ an alternative to the RGBA color space, such as CIElabor CIEluv, which are perceptually uniform color spaces and may bemore appropriate for representing quantitative abundances [36].Limitations of our definition of diversity. Our definition of diversitygeneralizes the one used in the field of Ecology to the case ofarbitrary multivariate data. As a consequence our definition looks atFig. 5. A real data set of 2550 college applicants with 10 attributesvisualized using the Diversity Map. Left: the subset of students recommendedfor acceptance based on a holistic admissions process implementedby a proprietary software package and designed to producea diverse incoming class. Right: the subset of rejected students. Therecommended students yield a visualization with a more even distributionof opacity, especially in attributes like GPA, ethnicity, residency, andmajor (columns 1, 4, 6, and 8 respectively). This suggests that the recommendedapplicants are more diverse than the rejected ones.the diversity of each attribute independently and does not take into accountthe interaction between attributes. In future work, we will investigatea definition to account for this interaction. The area of businessmanagement may provide useful insights as researchers in that fielddiscuss diversity across multiple attributes [19, 8].Application to Real Data. We also intend to explore the applicationof Diversity Maps to real-world data. As an example, we appliedDiversity Maps to a real data set containing 2550 applicants (one yearworth) to a particular university. Each applicant is characterized byten attributes. Interestingly, this real data set was preprocessed usingan existing proprietary software package designed to recommend a setof applicants using a holistic evaluation process intended to producea diverse incoming class. The DM visualizations of this data set areshown in Fig. 5. These results are promising in that they agree withthe output of the holistic evaluation software.We are also in the process of deploying an interactive version ofthe Diversity Map for use by ecologists at our university. Ecologistscommonly collect species inventory data and analyze it to better understandthe interactions between the environment and the species understudy. For example, species distribution modeling is used to relateenvironmental covariates to each species. Data sets of this type areoften challenging in many ways. In the Oregon State University H.J.Andrews Experimental Forest, researchers have collected data on 606moth species by sampling ∼200 sites across a ∼100 km 2 region ofstudy every summer week over a period of 23 years [24]. This data isinteresting because moths are indicators of broader biological diversityin plant types and physical environments. Diversity measures such asthe Shannon Index provide little insight into relationships that may bepresent in this data between the species and the environment because,in reducing diversity to a single number, they conceal a tremendousamount of information.8 CONCLUSIONSThe Diversity Map represents a first attempt to design a representationwith the specific goal of visualizing diversity as we have definedit in this paper. While, to date, little attention has been paid to thisproblem, we hope that this work will serve to provide a foundation forfuture studies into the design and evaluation of visualization methodsfor exploring the increasingly important area of diversity.ACKNOWLEDGMENTSThe authors wish to thank Onyekwere Ogba and Nicholas Hubbert fortheir development efforts supported in part by the CRA-W/CDC DistributedResearch Experiences for Undergraduates (DREU) program.In addition we would like to thank Dr. Margaret Burnett, Dr. JuanGilbert, Dr. Julia Jones, Dr. Jeff Miller, and Steven Highland for numerousdiscussions regarding our user study, application to admissionsdata, and application to moth diversity data. Finally we would like tothank our user study participants for their time and input. This workwas supported in part by NSF IIS-0546881.
Page 2 and 3: 1054 IEEE TRANSACTIONS ON VISUALIZA
Page 10: 1062 IEEE TRANSACTIONS ON VISUALIZA

Visualization of Diversity in Large Multivariate Data Sets

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?