Even as YouTube’s recommendation algorithm was rolled out with great fanfare, the fuse was already burning. A project of The Google Brain and designed to optimise engagement, it did something unforeseen — and potentially dangerous.
Today, we are all living with the consequences.
As Zeynep Tufekci, an associate professor at the University of North Carolina, explained to attendees of Hitachi Vantara’s Next 2019 conference in Las Vegas this week, “What the developers did not understand at the time is that YouTube’ algorithm had discovered a human vulnerability. And it was using this [vulnerability] at scale to increase YouTube’s engagement time — without a single engineer thinking, ‘is this what we should be doing?’”
The consequence of the vulnerability — a natural human tendency to engage with edgier ideas — led to YouTube’s users being exposed to increasingly extreme content, irrespective of their preferred areas of interest.
Tufekci is an expert in the fields of technology, privacy and surveillance, inequality, research methods and complex systems, who also writes for publications such as the New York Times and Wired.
In her keynote address, she explained that the idea of the recommendation engine seemed simple enough. “On YouTube, you watch something, you get recommended something else to watch, it auto-plays, and then you watch it or do not watch it, and then get recommended something else.”
She said the algorithm certainly increased engagement time significantly — which is what it was optimised to do.
However, she also uncovered an important unintended consequence when studying speeches given by Donald Trump in 2015 in the early days of his candidacy.
“I had taken notes of some of the things he was saying, but I wanted to get a quote he had said exactly right. So after having attended the rally I watched the same rally on YouTube, just to make sure I got the words exactly correct. I also checked a couple of other rallies to see if he had said it the same way or not.”
It was then that Tufekci started to notice YouTube was recommending some very extreme content for her.
“It started recommending scary things to me. I started getting autoplay recommendations for white supremacist kind of things, and then pretty soon it was recommending content saying the Holocaust never happened.
“I was thinking ‘whoa, what’s happening here, this is kind of scary.’ And I thought maybe this is a correlation — maybe there’s a subsection of this audience that’s watching these, and that’s what YouTube’s doing.”
She ran a series of experiments and started to find a consistent pattern. Watching speeches by Hillary Clinton or Bernie Sanders inevitably led to recommendations for more extreme left-wing content.
She then stepped beyond politics to see if the same thing was happening in other areas.
“I watched a video about being vegetarian, and YouTube was like ‘How about a video about being a vegan?’ So then I started watching videos about jogging and YouTube was like ‘How about an ultramarathon? Wouldn’t you like to run 36 miles?’ OK. It’s never hardcore enough for you.”
What Tufekci realised is that whatever she watched, YouTube would push her towards more extreme flavours of the content in the subject area of her choice.
A bug, not a feature
She told her audience this was not a design feature, but rather a consequence of allowing an algorithm to optimise around user engagement.
“What they had done was use machine learning to increase watch time. But what the machine learning system had done was to discover a human vulnerability. And that human vulnerability is that things that are slightly edgier are more attractive and more interesting.”
No less a figure than Facebook CEO Mark Zuckerberg has subsequently made the same point, she said. “When he was talking about extreme content, he said that algorithmically speaking what Facebook found was that content that pushes boundaries is more engaging. It kind of makes sense. When you think about people, we find things that are novel and exciting — [like] conspiracies — more fun. They’re more interesting than the basic truth.
“Imagine you’re a young person in Asia, and you’re devout, and you want to watch something about how to be a more devout Muslim. YouTube will recommend extremist content to you. If you are a young boy or a teenager, and you’re thinking ‘there’s feminism, but what about boys?’ You might ask some innocent questions, and YouTube will take that and lead you down an extreme path. Sometimes [it might be] white supremacy, or something very misogynistic, but will take people and push them to extremes automatically.”
According to Tufekci, the lesson from the YouTube example is that when people use these tools like machine learning — especially for insights — they need to recognise there are shadows there.
The problem is that developers often do not understand what the machine is optimising for.
“So we’re in Vegas. And if you want to find customers to come to Vegas, one of the things you probably don’t want to do, because it’s not ethical, is find compulsive gamblers, or identify people about to enter a manic depressive state — because people in the early stages of manic depressive disorder tend to become compulsive with spending money on gambling.”
Black boxes
Algorithms are black boxes, because we don’t really understand how they come to their insights, she said. “We don’t really know what they’re doing. We can just kind of see how they work. And they work sometimes surprisingly well.”
When developers are building these systems they are playing with a kind of intelligence. But it’s less like something they have constructed, and more than something they have grown in the garden.
“It’s kind of like having this child that you can try to influence. But in the end, machine intelligence does what it does. And that can have really dangerous consequences.”
The post YouTube’s algorithm hacked a human vulnerability, setting a dangerous precedent appeared first on Which-50.
