Unbiased computer confirms media bias

Hillary Clinton’s announcement last weekend that she is officially running for president set pundits spinning on both sides of the aisle. Released via a video on Clinton’s campaign website, the announcement featured only 92 words from the candidate, which were variously quoted by media outlets of all stripes. Consider this excerpt from The National Review, an outlet that self-identifies as conservative:

“I’m hitting the road to earn your vote because it’s your time, and I hope you’ll join me on this journey.”

Or this slice, featured by The Nation, which self-identifies as liberal:

“Everyday Americans need a champion, and I want to be that champion.”

From the quotes alone, you might not be able to tell whether the news outlet is liberal or conservative. But a computer probably can. Scientists developed an algorithm that, after churning through more than 200,000 quotes from 275 news outlets, discovered bias in their quote choice. Creating a graph that grouped media outlets by their selected quotes reveals pockets that pretty accurately reflect the political leanings of the outlets. The research suggests that information about an outlet’s political stripes is embedded in quote choice, surrounding context aside.

“Readers might experience very different personalities of the same politician, depending on what the news outlets they follow choose to quote from that politician’s public speeches,” says Cornell computer scientist Cristian Danescu-Niculescu-Mizil, a coauthor of the study, which in May will be presented at the World Wide Web Conference in Florence, Italy. Even though readers are exposed to the politician’s own words, “the part of the speech the reader has access to changes,” he says.

That might seem like a “no duh” conclusion. But the scholarship on media bias is fraught with its own biases, and for an objective algorithm to latch on to political leanings is a neat feat. Typically studies of bias start out by designating particular outlets as either left- or right-leaning and then examine, whether with a computer or human beings, how those biases play out. Danescu-Niculescu-Mizil and his colleagues took a different approach. They didn’t assign political slant, but just looked for patterns that emerge from the news outlet’s quoting behavior. They did this using the 2,274 speeches made by President Obama from 2009 to 2014 and 275 media outlets that quoted him.

The program then mapped quotes to the outlets in which they appeared, revealing a hidden media bias landscape. The researchers also created an algorithm, similar to those used by Netflix or Amazon to recommend new movies or books, that could predict with a fair level of certainty which quotes would end up in which outlets.

Many of the study’s take-home messages probably seem obvious. But when it comes to bias, we tend to underestimate our own and overestimate the enemy’s. Well, guess what: Almost all of us are wrong and right. Here are some of the fine grains:

1. Media bias exists. To quote their paper: “There is systematic bias in the quoting patterns of different types of news sources. We find that the bias goes beyond simple newsworthiness and space limitation effects, and we objectively quantify this.”

2.  In the mapped media bias landscape (imagine a square peppered with triangles representing media outlets) mainstream outlets are in a similar neighborhood, compared with their allegiance-declaring counterparts.

3. Within the two neighborhoods of mainstream and allegiance-declared outlets, conservative, liberal and international publications clump together (international outlets are closer to liberal outlets than to conservative ones). But despite this similar-stripe clumping, Fox News and The New York Times are relatively close to each other,suggesting that, as far as quoting patterns go, these two mainstream outlets aren’t all that different.

The speech corpus, which anyone can probe in nifty visuals, also uncovered some intriguing linguistic clues that you can keep an eye out for in coverage of speeches to come: Excerpts that used the word not or the contraction n’t are more likely to be quoted by mainstream conservative outlets. Danescu-Niculescu-Mizil and his colleagues speculate that because these negations are instances where the president contradicts or refutes a point, they might serve to highlight controversial aspects of the president’s remarks (the same might be true of liberal media with a conservative in the White House). There were also content-related cues: Quotes about the troops and war veterans were more common in conservative outlets, while financial and health care–related quotes were found more in liberal outlets.

Of course, quotes from speeches alone don’t tell you anything about the words surrounding those quotes, context that might reveal far more bias than the quotes themselves. But highlighting how quote choice fits into political coverage might help all of us become savvier media consumers. Now let’s see who quotes what from this post.