Fortune

MULLIGAN: BENJAMIN TICE SMITH; EX MACHINA: A24/COURTESY EVERETT COLLECTION; BATTLESTAR GALACTICA: FRANK OKENFELS—SCI-FI/PHOTOFEST;
THE MATRIX: WARNER BROS./PHOTOFEST; METROPOLIS: UFA/PHOTOFEST; ULTRON: WALT DISNEY STUDIOS MOTION PICTURES/COURTESY EVERETT COLLECTION
say, ‘Oh, now I can do accounting at home,’ ”
says Kate Crawford, principal researcher
at Microsoft and codirector of the AI Now
Institute at New York University. “These are
very advanced systems that are going to be
influencing our core social institutions.”
HOUGH THEY MAY not think of it as
such, most people are familiar
T with at least one A.I. breakdown:
the spread of fake news
on Facebook’s ubiquitous
News Feed in the run-up to the 2016 U.S.
presidential election.
The social media giant and its data scientists
didn’t create flat-out false stories. But the
algorithms powering the News Feed weren’t
designed to filter “false” from “true”; they were
intended to promote content personalized to
a user’s individual taste. While the company
doesn’t disclose much about its algorithms
(again, they’re proprietary), it has acknowledged
that the calculus involves identifying
stories that other users of similar tastes are
reading and sharing. The result: Thanks to an
endless series of what were essentially popularity
contests, millions of people’s personal
News Feeds were populated with fake news
primarily because their peers liked it.
While Facebook ofers an example of how
individual choices can interact toxically with
A.I., researchers worry more about how deep
learning could read, and misread, collective
data. Timnit Gebru, a postdoctoral researcher
who has studied the ethics of algorithms at Mi-
A.I. SPECIAL REPORT
machine-learning
algorithms “haven’t
been optimized for any
definition of fairness.
They have been
optimized to do a task.”
> deirdre mulligan
associate professor, uc berkeley school of information
crosoft and elsewhere, says she’s concerned about how deep learning
might afect the insurance market—a place where the interaction of
A.I. and data could put minority groups at a disadvantage.
Imagine, for example, a data set about auto accident claims. The
data shows that accidents are more likely to take place in inner
cities, where densely packed populations create more opportunities
for fender benders. Inner cities also tend to have disproportionately
high numbers of minorities among their residents.
A deep-learning program, sifting through data in which these
correlations were embedded, could “learn” that there was a relationship
between belonging to a minority and having car accidents, and
could build that lesson into its assumptions about all drivers of color.
In essence, that insurance A.I. would develop a racial bias. And that
bias could get stronger if, for example, the system were to be further
“trained” by reviewing photos and video from accidents in inner-city
neighborhoods. In theory, the A.I. would become more likely to conclude
that a minority driver is at fault in a crash involving multiple
drivers. And it’s more likely to recommend charging a minority
driver higher premiums, regardless of her record.
... to malevolent
Or will they develop their own agendas, in pursuit of a colder logic that sees us as a distraction or a threat? It’s the “intelligence” of A.I., the facet
that makes a computer almost human, that compels us to ask these questions again and again. —Matt Heimer and Armin Harris
www.t.me/velarch_official
AVA
Ex Machina
NUMBER SIX
Battlestar
Galactica
AGENT SMITH
The Matrix
FALSE MARIA
Metropolis
ULTRON
Avengers:
Age of Ultron
pure evil
59
FORTUNE.COM // JULY.1.18