Practitioners-Guide-User-Experience-Design

RELEASE, FIX, RELEASE
There is one last type of testing to discuss. The kinds of user tests I’ve covered are the
next best thing you can do to actually shipping the product, and some leading web
companies have famously made a practice of going ahead and doing testing by shipping,
releasing their products to the wild. Shipping products has become an audacious form of
prototyping.
Facebook has been one big advocate of this, and you could say the company is in a
state of perpetual prototyping. According to the Mashable article “Facebook Launches
Redesigned Mobile App for iOS 7,” the company did six months of testing leading up to
the final release of its iOS7 redesign, in which “1 to 2 million users [were] unknowingly
utilizing versions of the redesigned navigation bar.” 4 Such a large test is known as a
bucket test. Fully functional versions of an app or feature are tested on the broad public.
Google also has a long history of publicly testing new products in beta versions. Gmail
was one such innovation, and it was in beta for five years, being tested and updated until
the beta label was removed in 2009 after developers integrated security improvements,
document viewing, increased storage space, and a mobile-optimized version.
This may be the best way to get lots and lots of user feedback fast, and it can be
viewed as an especially careful way of going about innovation because it allows for
testing with so many users. But it also clearly has its dangers, prime among them being
that it often means failing publicly. The UX leaves a lot to be desired, and many products
from both of these companies have had less than celebratory receptions at release. Many
innovative startups have also espoused a “Fuck it, ship it” approach, opting to constantly
update their products, with no one version ever being considered final. These ways of
operating are based on a credo of “failing fast,” getting actual products out there and then
quickly updating them. The companies that advocate this way of operating, and its public
failure, can do so in part because they have big development teams, which means they can
fix fast as well. For most companies, failing privately, testing on smaller groups of users,
is the way to go. Steve Krug has argued that testing with just three to five real users will
catch 80 percent of usability issues. I know I’ve found that my quick and dirty hallway
tests have been invaluable, and most often this smaller-scale testing that you initiate on
your own is going to be your best means of getting real user feedback all the way through
your design process.
Knowing when and how to innovate is a hallmark of good UX practice. You should
always strive to move products forward and to bring your users along on that path to the
future. But you’ve also got to get used to leaving lots of your ideas on the cutting room
floor. Test early and test often, arm yourself with the best user data and analytics you can,
and always be prepared to put an innovation on hold. Chances are good you’ll be able to
come back to it, and with the rate of evolution of digital products, you may not have to
wait long.