Fortune

A.I. SPECIAL REPORT
W
INVISIBLE Timnit
Gebru has studied
the ways that A.I. can
misread, or ignore,
information about
minority groups.
WHEN TAY MADE HER DEBUT in March 2016, Microsoft had high hopes
for the artificial intelligence–powered “social chatbot.” Like the automated,
text-based chat programs that many people had already
encountered on e-commerce sites and in customer service conversations,
Tay could answer written questions; by doing so on Twitter
and other social media, she could engage with the masses.
But rather than simply doling out facts, Tay was engineered to
converse in a more sophisticated way—one that had an emotional
dimension. She would be able to show a sense of humor, to banter
with people like a friend. Her creators had even engineered her to
talk like a wisecracking teenage girl. When Twitter users asked Tay
who her parents were, she might respond, “Oh a team of scientists in
a Microsoft lab. They’re what u would call my
parents.” If someone asked her how her day had
been, she could quip, “omg totes exhausted.”
Best of all, Tay was supposed to get better
at speaking and responding as more people
engaged with her. As her promotional material
said, “The more you chat with Tay the smarter
she gets, so the experience can be more personalized
for you.” In low-stakes form, Tay was
supposed to exhibit one of the most important
features of true A.I.—the ability to get smarter,
more efective, and more helpful over time.
But nobody predicted the attack of the trolls.
Realizing that Tay would learn and mimic
speech from the people she engaged with,
malicious pranksters across the web deluged
her Twitter feed with racist, homophobic, and
otherwise ofensive comments. Within hours,
Tay began spitting out her own vile lines on
Twitter, in full public view. “Ricky gervais
learned totalitarianism from adolf hitler, the
inventor of atheism,” Tay said, in one tweet
that convincingly imitated the defamatory,
fake-news spirit of Twitter at its worst. Quiz
her about then-president Obama, and she’d
compare him to a monkey. Ask her about the
Holocaust, and she’d deny it occurred.
In less than a day, Tay’s rhetoric went from
family-friendly to foulmouthed; fewer than 24
hours after her debut, Microsoft took her ofline
and apologized for the public debacle.
What was just as striking was that the wrong
turn caught Microsoft’s research arm of guard.
“When the system went out there, we didn’t
plan for how it was going to perform in the
open world,” Microsoft’s managing director of
research and artificial intelligence, Eric Horvitz,
toldFortune in a recent interview.
After Tay’s meltdown, Horvitz immediately
asked his senior team working on “natural
language processing”—the function central
to Tay’s conversations—to figure out what
went wrong. The staf quickly determined
that basic best practices related to chatbots
were overlooked. In programs that were more
rudimentary than Tay, there were usually
protocols that blacklisted ofensive words, but
there were no safeguards to limit the type of
data Tay would absorb and build on.
Today, Horvitz contends, he can “love the
example” of Tay—a humbling moment that
Microsoft could learn from. Microsoft now
deploys far more sophisticated social chatbots
www.t.me/velarch_official
CODY O’LOUGHLIN: THE NEW YORK TIMES— REDUX; PREVIOUS SPREAD, STATUE: ARTNELI/ALAMY
56
FORTUNE.COM // JULY.1.18