The Windows@ 95 User Interface: Iterative Design and ... - Microsoft

-_-_-_I,, _ _I A ; _-__ --__ ._._ IIThe Windows@ 95 User Interface: Iterative Design and ProblemTracking in ActionKent SullivanMicrosoft CorporationOne Microsoft WayRedmond, WA 98052-6399 USA+12069363568kentsu@microsoft.comABSTRACTThis organization overview discusses the lessons learned inapplying the usability engineering principles of iterativedesign and problem tracking to the development ofWindows 95, a large commercial software project.CONTEXTWindows 95 is an extensive upgrade to the Windows 3.xfamily of operating systems. The user interface is but oneof many areas of the product which featured dramaticchanges.based on descriptions of usability engineering in [l] and[3]. We began by creating paper or computer-basedprototypes. Each prototype was used to try out design ideasand to gather usability data in the lab. After testing in theusability lab, the prototype was usually improved and retested.Once results were satisfactory, the design was codedthen refmed in the usability lab. Following lab testing, weexamined the product as a whole over time in the field(during beta testing).The user interface design team formed in October, 1992,near the beginning of the project, and was composed ofapproximately twelve people skilled in product design,graphic design, usability testing, and computer science. Thesoftware developers who implemented the UI accounted foranother twelve or so people. The entire Windows 95 teamwas over 400 people.FOCUSThe design team was chartered with two very broad goals:• Make Windows easier to learn for people just gettingstarted with computers and Windows.0 Make Windows easier to use for people who alreadyuse computers-both the typical Windows 3.x user andthe advanced, or “power”, Windows 3.x user.Having to design for these two widely-different groups ofusers told us that this project was not going to be a trivialundertaking. Without careful design and testing, we werelikely to make a product that improved usability for someusers and worsened it for millions of other users (existingor potential).APPROACHOur approach was anchored by the two usabilityengineering principles of iterative design and problemtracking.Iterative DesignFigure 1 outlines the iterative design process that we used,Permission to make digitalhard copies of all or part ofthis material forPersonal or chwoom use is Sronted without fee provided that the copiesmo not made or distributed for profit or commercial advantage, the copyrightnotice, the title of the publication ‘and its date appear, ad notice isk’en that coptight is by permission ofthe ACM, Inc. To copy otherwise,to republish, to post on servem or to redistribute to lists, requirespecificpermission and/or feeICSE 97 Boston MA USACqW-$$t 1997 ACM O-89791-914-9/97/os ..$3.5056;I ’I I -Figure 1: Windows 95 iterative design process.We conducted 64 phases of lab testing, using 560 subjects.50% of the users were intermediate Windows 3.x users; therest were beginners, advanced users, and users of othersystems. Examples of designs we iterated and results fromusability testing we conducted are documented in [2].Problem TrackingI created a relational database to enforce a standard way ofrecording all the usability issues, when and how they werefured, and whether issues remained after re-testing withusers. Over the course of the project, 551 problems wererecorded. Each problem was rated with one of three levelsof severity. 15% were judged to be level 1 (most serious),43% level 2, and 42% level 3.Problems could be resolved in one of three ways:completely, partially, or not at all. By the end of theproject, 81% of the problems were completely addressed,8% were partially addressed, and 11% were unresolved.CURRENT STATEWindows 95 shipped in August, 1995 and has sold 40million units in its fast year. Windows NT 4.0 (shipped oneyear later) inherited the Windows 95 interface withminimal changes. The Windows 95 user interface serves asthe base for further products, such as the integration ofWorld-Wide Web browsing into the desktop found inInternet Explorer 4.0.LESSONSAlthough the design process we employed is widelyaccepted and fairly straightforward, we learned a number of

important lessons about how to best apply it. We alsolearned many things that changed the product we werecreating. Three of the most important ones are discussedbelow.Importance of baseline with existing productThe initial design for the desktop and file system UI wasbased on Windows 3.x and work done by the “Cairo”*team. These predecessors incorporated concepts validatedin even earlier systems such as the Apple Macintosh andXerox Star. Usability testing of the initial Windows 95design yielded fairly negative results. Both beginning andintermediate users had difficulty understanding basicconcepts in the new system. The magnitude of these resultswas surprising, because we thought we were on the righttrack by making small changes to the existing UJ.We realized that we needed a baseline with Windows 3.x,to better understand the magnitude of the prior problemsand what problems were unique to the new design. First,we gathered market research data about Windows 3.xusers’ top 20 most frequent tasks. We then conductedseveral lab studies comparing the existing and new designs,focusing on the top 20 tasks. The results were veryenlightening and greatly changed the design of the product.Had we begun the project with a baseline test of Windows3.x, considerable time and energy would have been saved.Scalability: the agreeable middle groundIn the initial design for Windows 95, we were afraid ofgoing too far and alienating the existing user base. Resultsfrom the baseline testing encouraged us to experiment withradical solutions. We created a separate UJ for beginnersand mocked it up. Usability testing proved successful,because actions were very constrained, but limitations withthis approach quickly appeared. For example, if just onefunction a user needed was not supported in the beginnershell, s/he would have to abandon it (at least temporarily).For these and other reasons, we abandoned the separateshell idea.After our two design failures, it became clear to us that atruly usable system would scale to the needs of differentusers: it would be easy to discover and learn yet wouldprovide efficiency (through shortcuts and alternatemethods) for more-experienced users. Scalability is visiblethroughout the final product, especially in the Start Menu.Replacements for the monolithic design specDuring the first few months of the project, our design specgrew by leaps and bounds and reflected hundreds ofperson-hours of effort. However, due to the problems wefound via user testing, the design documented in the specwas suddenly out of date. We faced a major decision: spendweeks changing the spec to reflect the new ideas and losevaluable time for iterating or stop updating the spec and letthe prototypes and code serve as the spec.1A future version of Microsoft Windows N?TM. Initialdesign work for the Cairo UI began in 1990.After some debate, the team decided to take the latterapproach. While this change made it somewhat moredifficult for outside groups to keep track of what we weredoing, it allowed us to iterate at top speed. The change alsohad an unexpected effect: it brought the whole team closertogether because much of the spec existed in conversationsand on white boards in people’s offices. Many “hallway”conversations ensued and continued for the duration of theproject.To ensure that interested parties stayed informed about thedesign, we:1.2.3.4.Held regular staff meetings for the design team. Heldweekly, these meetings allowed each of us to check inand to quickly learn how our work affected what otherswere doing.Broadcasted usability test schedules and results viaelectronic mail. Team members found always had thelatest test schedules and testing results handy.Formally tracked usabilitv issues, as discussedpreviously.Held regular design presentations for outside groups.Presenting to groups “live” was more effective than awritten document, because the presentations invitedricher feedback and always contained up-to-dateinformation.The “prototype or code are the spec” approach overallworked well, although we naturally have refined theprocess over time. For example, all the prototypes for agiven release of the product now reside in a commonlocation on the network and include instructions forinstalling and running them. The design team continues towrite initial specification documents and circulate them forearly feedback. Once prototyping and usability testing hasbegun, however, the spec often refers readers to theprototype for details. This, in turn, invites richer feedback.Using a prototype or code as the spec may not work well inother types of projects, such as contracted or procuredsoftware development, where completed specs are oftenrequired up front as part of a project’s contract.REFERENCES1. Nielsen, J. (1993). Usability Engineering. San Diego,CA: Academic Press, Inc.2. Sullivan, K. (1996). The Windows 95 User Interface:A Case Study in Usability Engineering. In CHI 96Conference Proceedings (Vancouver, BC, May 1996),ACM Press, 473-480.3. Whiteside, J. L., Bennett, J, & Holtzblatt, K. (1988).Usability Engineering: Our Experience and Evolution.In M. Helander (Ed.), Handbook of Human-ComputerInteraction (pp. 791-817). Amsterdam: ElsevierScience Publishers, B. V.563