The Role of Eye Tracking in User Experience Research

hfes.org
  • No tags were found...

The Role of Eye Tracking in User Experience Research

The Role of Eye Tracking inUser Experience ResearchHFES Webinar SeriesPrepared by User Centric, Inc.Gavin LewManaging Directorglew@usercentric.comPresentation Delivered11 December 2009


OutlineThe BasicsEye Tracking and UX ResearchThe Case for Quantitative AnalysisCeilings and FloorsMethod case study on packagesQ & A2


Eye TrackingThe Basics3


Basics of Eye MovementsEye Tracking: The BasicsEye tracking is aresearch techniquethat captures eyebehavior in responseto a visual stimulus“Eye-mindhypothesis”: wherepeople look is wherethey focus theirattentionSaccadic eyemovements (mostcommon) consist of:– Fixations– SaccadesFixation (circle)Saccade (line)4


Sample Eye MeasuresEye Tracking: The BasicsQuantitative Measures# fixations on an areaFixation length# fixations before targetTime to 1 st target fixationScanpath complexity% users fixating on an areaOrder of 1 st fixation# visits to areaPupil diameterMeaningInformativeness of an area / user interest in theareaInfo clarity / info density / info processingdemandsLayout effectiveness / search demandsProminence / perceived importance of an areaCognitive processing demands / user mentalworkload / emotion5


Eye TrackersEye Tracking: The BasicsEye tracker determines the position of one or both eyes multipletimes (30 – 1000+) per secondCommonly used eye trackers differ in physical form, setupprocedures, and tracking methodology:6


The Tobii Eye TrackerEye Tracking: The Basics Our eye tracker: Tobii 1750– Integrated into a 17”computer monitor– No restraints, freedom ofhead movement– Binocular tracking– Sampling rate 50 Hz– Quick and automaticcalibration7


Lab SetupModerator’s station to vieweye gaze in real time andcontrol eye tracking softwareFacecameraEye Tracking: The BasicsTobii 1750 remote eye-trackingsystem integrated into a 17”monitor (set to 1024 x 768 px)One-waymirrorModeratorParticipant8


Eye TrackingEye Tracking and UX Research9


Three Dimensions of FeedbackEye Tracking: Eye Tracking and UX ResearchHow people evaluate objects− Commercials− Packages− Online advertisements− Products− …© December 11, 200910


AttitudeEye Tracking: Eye Tracking and UX Research What users “say” ...Influencers– Social status– Emotion– Coolness / HipReveal– Feature importance– Purchase intent© December 11, 200911


BehaviorEye Tracking: Eye Tracking and UX ResearchWhat users actually “do”Ultimately, behaviors arewhat we wish to shapeGive users context, a taskand stimuli and then– Observe what users doBehavior drives usage© December 11, 200912


AttentionEye Tracking: Eye Tracking and UX ResearchWhat users “focus” onWhat happens inside the headSometimes users are unawareOften attention measured byeye tracking© December 11, 200913


Eye Tracking: Eye Tracking and UX ResearchEye Tracking in UX Research14


Eye Tracking: Eye Tracking and UX ResearchHow Can Eye Tracking Help Usability Testing?TASK: Find a branch near youPage with nogaze. Withsomething hardto find.That’simpossible.15


Eye Tracking: Eye Tracking and UX ResearchHow Can Eye Tracking Help Usability Testing?ET FINDINGS:(1) Average # fixations before the targetlink was found: 112(2) The top right corner of the pageattracted initial & the most fixations.Ah, quant data…16


Eye Tracking: Eye Tracking and UX ResearchHow Can Eye Tracking Help Usability Testing?TASK: Find a list of indoor climbing walls.So… she onlyconsidered thefirst two results?17


Eye Tracking: Eye Tracking and UX ResearchHow Can Eye Tracking Help Usability Testing?ET FINDING: She looked at (considered) moreresults than just the top 2.18


Eye Tracking: Eye Tracking and UX ResearchHow Can Eye Tracking Help Usability Testing?Typically in usability testing, wecollect:– Behavioral measures (e.g.,clicks, time)– User self-report (e.g., RTA)Eye tracking can:– Support and illustrate UTfindings– Help determine userexpectations– Augment usability findingsby filling in the gapsbetween• Observable events• User comments19


Eye Tracking: Eye Tracking and UX ResearchHow Can Usability Testing Help Eye Tracking?Different eye movement patternsproduced by the same personlooking at the same picture…Why are theyall so different?Yarbus, A. L. Eye Movements and Vision. Plenum. New York. 196720


Eye Tracking: Eye Tracking and UX ResearchHow Can Usability Testing Help Eye Tracking?TASK: Estimatepeople’s ages.Different eye movement patternsproduced by the same person lookingat the same picture…but in a different context!TASK: Estimate thefamily’s materialcircumstances.Yarbus, A. L. Eye Movements and Vision. Plenum. New York. 196721


Eye Tracking: Eye Tracking and UX ResearchHow Can Usability Testing Help Eye Tracking?TASK: Find the museum hours.22


Eye Tracking: Eye Tracking and UX ResearchHow Can Usability Testing Help Eye Tracking?ET FINDINGS:(1) All users looked at the image.(2) Time spent looking at the image:30% of all time spent on page.Great!Image got lotsof attention!23


Eye Tracking: Eye Tracking and UX ResearchHow Can Usability Testing Help Eye Tracking?Actual target linkSo this isbad…24


Eye Tracking: Eye Tracking and UX ResearchUsability Testing and Eye TrackingUsabilityTestingreveals (mostly)outcomes ofinteractionProvides context for…Illustrates and helps understandEye Trackingreveals factorsthat contributeto the outcomesANSWERSPRACTICALQUESTIONSPROVIDESMOREDETAILEDANALYSIS25


Eye Tracking: Eye Tracking and UX ResearchEye Tracking In Isolation Yields UncertaintyEye tracking has limitedapplicability when used inisolation– A fixation on a face mayindicate recognition, liking,dislike, or confusion– More fixations may indicateinterest or inefficient searchInstead combine withattitudinal and behavioralprobes– Learn the “why”– Just knowing where peoplelook is often insufficient26


Eye Tracking: Eye Tracking and UX ResearchContext is Key for Eye TrackingIs this good or bad?Were users told to:– Signup for Newsletters?– Learn about services?– Read publications?– Look for a job? Or worse, they were not givenany context at allAlways ask aboutthe context / task!© December 11, 2009 – Proprietary and Confidential 27


Eye TrackingThe Case for Quantitative Analysis28


Eye Tracking: The Case for Quant AnalysisHeatmap: Classic Eye Tracking OutputWith thousands of datapoints to analyze,interpretation should bebased on data…© December 11, 2009 – Proprietary and Confidential 29


Eye Tracking: The Case for Quant AnalysisVisualizations Facilitate, Not Interpret© December 11, 2009 – Proprietary and Confidential 30


Eye Tracking: The Case for Quant AnalysisGraphs are Visualizations too… Show Me Data© December 11, 2009 – Proprietary and Confidential31


Eye TrackingThe Case for Quantitative Analysis• Case Study32


Selecting One DesignEye Tracking: The Case for Quant AnalysisConcept 1Concept 3Existing HomepageConcept 2Concept 4© December 11, 2009 – Proprietary and Confidential 33


Free View: First ImpressionsClient identified key businessgoalsEye Tracking: The Case for Quant AnalysisMY ACCOUNTBetween-groups design– Each group saw a differentdesignPERSONAL/BUSINESSPROMO1 st impressions task:– You are looking for a newwireless provider and youdecided to check whatVerizon Wireless has tooffer... Homepage was shown for 10sec.© December 11, 2009 – Proprietary and Confidential 34


Which Won?Eye Tracking: The Case for Quant AnalysisCONCEPT 1CONCEPT 2 CONCEPT 3 CONCEPT 4 EXISTINGRemember themeasures? Whatis the heatmapshowing?!?!Sample Eye MeasuresQuantitative Measures MeaningEye Tracking: The BasicsCONCEPT 1% of users whofixated?Fixation length?# of fixations?CONCEPT 2 CONCEPT 3 CONCEPT 4 EXISTING# fixations on an areaFixation length# fixations before targetTime to 1 st target fixationScanpath complexity% users fixating on an areaOrder of 1 st fixation# visits to areaPupil diameterInformativeness of an area / user interest in theareaInfo clarity / info density / info processingdemandsLayout effectiveness / search demandsProminence / perceived importance of an areaCognitive processing demands / user mentalworkload / emotionOh, wait. Maybethis will help36© December 11, 2009 – Proprietary and Confidential 35


You Must Know the Measure!Eye Tracking: The Case for Quant Analysis% users who fixated each areaCONCEPT 1CONCEPT 2 CONCEPT 3 CONCEPT 4 EXISTINGOrder in which the areas were first visitedCONCEPT 1CONCEPT 2 CONCEPT 3 CONCEPT 4 EXISTING© December 11, 2009 – Proprietary and Confidential 36


Quantitative ResultsEye Tracking: The Case for Quant Analysis% users who fixated each areaCONCEPT 1CONCEPT 2CONCEPT 3CONCEPT 4EXISTINGKEY AREAS OF INTERESTACBCBACABCABBCAAREA A: My Account100%79%77%64%68%AREA B: Promo100%100%100%100%100%AREA C: Personal/Business100%85%80%100%52%Order in which the areas were first visitedCONCEPT 1CONCEPT 2CONCEPT 3CONCEPT 4EXISTINGKEY AREAS OF INTERESTAREA A: My Account2 nd>5 th> 5 th3 rd4 thAREA B: Promo3 rd1 st4 th2 nd1 stAREA C: Personal/Business1 st2 nd1 st1 st3 rd© December 11, 2009 – Proprietary and Confidential 37


Quantitative ResultsEye Tracking: The Case for Quant Analysis# of fixations on an area (out of ~31 on average)CONCEPT 1CONCEPT 2CONCEPT 3CONCEPT 4EXISTINGKEY AREAS OF INTERESTAREA A: My Account3.83.42.32.22.3AREA B: Promo8.28.05.56.58.3AREA C: Personal/Business8.24.23.56.93.2TOTAL 20.215.6 11.315.6 13.8© December 11, 2009 – Proprietary and Confidential 38


Bottom LineLink measures to researchquestionEye Tracking: The Case for Quant AnalysisMY ACCOUNTWhen looking at data, avoidthe allure of reading tealeavesPERSONAL/BUSINESSPROMOAsk:– What is the context/task?– What is the measure?– Where is the quant?© December 11, 2009 – Proprietary and Confidential 39


Eye TrackingCeilings and Floors40


Commercial aired in 2003Eye Tracking: Ceilings and FloorsFYI: Commercial is for a cable TV, Internet and telephone provider41


What Did You See?Eye Tracking: Ceilings and FloorsServices offeredInternet web pageMoney in a piggy bankLet’s see what Participant #7 saw…42


Participant #7’s Gaze ReplayEye Tracking: Ceilings and Floors43


What Did Participant #7 See?Eye Tracking: Ceilings and FloorsKey wordsWatched pigEven followed the mouse!In testing, we asked about the commercial…– Love it– I want the dealSo, you go and get it!– Literally. Participant asked to open a browser and off they went– They even entered the URL…44


Eye Tracking: Ceilings and FloorsTV Commercial to Web “Handshake”Empirical weblog data:– 20% clicked the pig45


ResultsEye Tracking: Ceilings and FloorsAttitude– Viewers liked the ad– Some even “loved it” and wanted to get the “deal”Behavior– In the study, 23% clicked (empirically, 20% click)Attention– 70% lookedNote how EyeTracking was notused in isolation…• But, the pig spins around, so some glances will occur– Setting a 150ms threshold (look at pig, read, etc.)• 40% looked• Relative to ad spaces on site46


Ceilings and FloorsEye Tracking: Ceilings and FloorsNot everything you design gets attention (there is a “ceiling”) Designers and marketers tend to assume that if you build it, 100%will see itWhat would you say about these results?We said:– Not bad. Your ceiling was 40% and you are getting 20%.– To improve click-thru, improve attention.• If you do not attend, you will not click.Eye Tracking is useful answer: Do user NOT CLICK because– They looked, but the value proposition click (marketing).– They never looked (design).47


Eye TrackingComparing Search Results: Bing vs. Google48


Eye Tracking: Bing vs. GoogleComparing Two Search Results PagesResearch objective:– Compare the distribution of attention on equivalent areas of Bingand Google– Assess how much attention is captured by elements that areunique to BingParticipants (N=21) completed search tasks for each engine– Two informational (e.g., "Learn about eating healthy")– Two transactional (e.g., "Book a last minute vacation“)49


Gaze Duration for One TaskEye Tracking: Bing vs. Google© December 11, 2009 – Proprietary and Confidential50


Results: SimilaritiesEye Tracking: Bing vs. GoogleGoogle and Bing did not differ inamount of attention on the organicsearch results– In each search, all participantslooked at the organic searchresults, spending an average of 7seconds in that areaAttention on the sponsored linkslocated above the organic results wassimilarly high for both Bing andGoogle– Over 90% of participants looked inthat area during each search.51


Results: DifferencesEye Tracking: Bing vs. GoogleSponsored links on the right attractedmore attention– Bing (~42% of participants persearch) than they did on Google(~25% of participants per search)– Participants who fixated on theselinks spent approximately 2.5seconds looking at the area duringtransactional searches and 2seconds during informationalsearches52


Eye TrackingMethod Case Study on Packages53


Method Case StudyEye Tracking: MethodStudy objectives:– Determine how the client’s packagingcompares to competitors’ packaging interms of noticeability and visualengagement / interest– Assess the findability and clarity of theinformation on the client’s package ascompared to the information on thecompetitors’ packagesMarketingUsabilityWearable eye trackers vs. remote eye trackers54


A Walk Down The Aisle…Macro-level:– Participants saw pictures ofaisles– Five picture frames perstore (2 s per frame)– Each showed the entireprepaid phones display– However, the display was ina different position in eachframe to simulate thechanging view of a movingcustomer (in a “freezeframe” way) and minimizethe effect of product positionin the pictureEye Tracking: Method55


Product ViewMicro-level:– Participants saw pictures ofindividual packages– Given tasks to answer to findinformation on the packageitself– Participants were able to“flip” package around byclickingFrontEye Tracking: MethodBackQ2: What isthe brand ofthis phone?The answer is:______Q3: Can youbrowse theWeb on thisphone?The answer is:______56


Actual Package ExperienceConstructed shelvesEye Tracking: MethodParticipants were different fromthe macro- and micro-levelstudy– Far view– Up close– Manipulation– Selection decision– Discussion57


MeasuresMemory– Free recall of products and brands– Recognition exercises of products and brandsEye Tracking: MethodPreference– First impressions from afar– Likes and dislikesPerformance tasks– Accuracy of finding answers to tasks– Efficiency in time to find answers (excluded incorrect answers)Eye tracking (computed based on package area)– % who looked (noticeability)– # of fixations (visual engagement)58


Eye TrackingQ & A59


Final TakeawaysQ &AEye Tracking should be combined with other UX techniques– Avoid using ET in isolationContext matters– One can change a heatmap with just four words!When looking at a heatmap, ask:– What was the task or context?– What is the heatmap showing?• % who fixated, gaze duration• # of fixations, etc.• Each measure reveals a different story…With so much data, where is the quantitative analysis?60


Q & A?Q &AQuestions?Additional references:– Me: glew@usercentric.com– Peer-reviewed journal articles, presentations,white papers on www.usercentric.com– ET case study of package labels by Aga Bojkoin Tullis and Albert’s book, Measuring TheUser Experience– ET global case study in Bob Schumacher’sbook, The Handbook of Global Research• www.globaluserresearch.com• www.elsevierdirect.com61

More magazines by this user
Similar magazines