How do you tell when the news is biased? It depends on how you see yourself (by Jonathan Stray)

Jonathan might be Stray, but he's certainly not LOST.


How do you tell when the news is biased? It depends on how you see yourself

Does the quest for balance in news stories open journalists up to claims of bias? It’s all about the framing.

By Jonathan Stray

Email

Take a moment with the headlines from this screenshot of The New York Times homepage from January. Really — it’s a little experiment. Click the image above for a larger view if you need to.

Ready?

How did you feel about these headlines? Does it matter to you to learn that they actually came from Fox News on the same day? (Screenshot for proof.) This faux home page was created by Dan Schultz, the MIT grad student also responsible for Truth Goggles, using his NewsJack point-and-click “remixer.”

Knowing what you know now, do these headlines seem different to you? If so, you’ve just proved that we detect and judge bias based on things other than what journalists actually write.

This effect has been noticed before. At the University of Michigan, William Youmans and Katie Brown showed the same Al Jazeera English news clip to American audiences, but with a catch: Half saw the news with its original Al Jazeera logo intact, and half saw the same video with a CNN logo instead. Viewers who saw the story with the original Al Jazeera logo rated Al Jazeera as more biased than before they had seen the clip. But people who watched the same footage with the fake CNN logo on it rated CNN as less biased than before!

Does this mean that we judge “bias” by brand, not content? Many people have tried to define what media bias is, and attempted to measure it, but I want to try to answer a different question here: not how we can decide if the news is biased, but how each of us actually does decide — and what it means for journalists.

The hostile media effect

During the Lebanese civil war in 1982, Christian militias in Beirut massacred thousands of Palestinian refugees while Israeli solders stood by. In 1985, researchers showed television news coverage of the event to pro-Israeli and pro-Arab viewers. Both sides thought the coverage was biased against them.
This effect — where both sides feel that a neutral story is biased against them — has been replicated so many times, in so many different cultural settings, with so many types of media and stories, that it has its own name: hostile media effect. The same story can make everyone on all sides think the media is attacking them.

Like a lot of experimental psychological research, the hostile media effect suggests we’re not as smart as we think we are. We might like to think of ourselves as impartial judges of credibility and fairness, but the evidence says otherwise. Liberals and conservatives can (and often do) believe the same news report is biased against both their views; they aren’t both right.

But why does this happen? Specifically, why does it happen for some stories and topics and not others? Discussion of climate change often provokes charges of bias, but discussion of other hugely significant science stories, such as the claimed link between vaccination and autism, usually produces a much smaller outcry.

You see bias when you see yourself as part of a group

Communications researcher Scott Reid has proposed that we can explain the hostile media effect through the psychological theory of self-categorization. This is a theory about personal identity and group identity, and it says that we “self-stereotype,” placing conceptual labels on ourselves just as we might make assumptions about other people. We all have multiple identities of this kind: gender, age, political preferences, race, nationality, subculture, and so on.

To test this, he performed a series of recently published experiments with American students. In the first, he used a survey to ask people whether they thought the media was biased, as well as their personal political orientations, both on a numerical scale from liberal to conservative. The catch was different groups got different cover pages with different sets of instructions. The first set of instructions was neutral:

The purpose of this questionnaire is to get your views of the news media in general.

The second set of instructions was designed to play up feelings of partisanship:

In recent times the differences between Republicans and Democrats have become highly polarized. Many of the issues discussed in the media are seen very differently by Republicans and Democrats. In this context, it is important to gauge people’s views of the media.

The third set of instructions was also designed to reinforce an identity, but in this case an identity that might be common to both liberals and conservatives — that of being an “American” versus the rest of the world.

With increasing globalization, it has become apparent that the media differs across countries and cultures. Al Jazeera has become the voice for much of the Arab world, both within the United States and in the Middle East. Given these changes, it is important to gauge people’s views of the news media in the United States.

And, oddly enough, the same survey gave different results, depending on the instructions:

Each of the lines on this graph shows how people’s perception of bias varied with their political orientation. The downward slope means that the more conservative someone was — the farther to the right on the “political position” scale — the more they perceived the media as hostile to Republicans, just as expected.

The surprising thing is that the strength of this perception depended on the framing each group had been given. When people were prompted to think about Republicans and Democrats, they perceived more media bias against their views, as indicated by the steep dashed line. When they were instructed to think about America vs. the world, they perceived slightly less bias then the neutral condition, as indicated by the shallow dotted line. Our perception of bias changes depending on the self-identity we currently have in mind.

This self-categorization explanation also predicts that people who are more partisan perceive greater bias, even when the news is in their favor. In Reid’s second experiment, people read an article about polling numbers for the 2008 presidential primaries, containing language like “among Republicans, former New York mayor Rudy Giuliani maintained a 14-point lead over Arizona Sen. John McCain for the Republican presidential nomination,” and similar statements about the Democratic candidates. This time, the source of the information was manipulated: One group saw the poll attributed to the “Economic Policy Institute, a Democrat think tank and polling agency,” while the other was told it came from the “American Enterprise Institute, a Republican think tank and polling agency.”

In this purely factual scenario — dry-as-toast poll numbers, no opinions, no editorializing — respondents still had completely different reactions depending on the source. As you might expect, people who believed that the poll numbers came from the American Enterprise Institute thought that the story was biased towards Giuliani (and vice versa), confirming the hostile media effect. But the perception of favoritism increased not according to whether the reader personally identified as Republican or Democratic, but on how strong this identification was. The implication is that if you feel strongly about your group, you’re likely to see all news as more biased — even when the bias favors you.

Reid’s final experiment tested perceptions of overt attacks. He used a scathing review of Michael Moore’s Fahrenheit 911, originally published on Slate, which begins:

One of the many problems with the American left, and indeed of the American left, has been its image and self-image as something rather too solemn, mirthless, herbivorous, dull, monochrome, righteous, and boring.

The copy given to subjects (falsely) claimed the author was a member of either a Democrat or a Republican think tank. (In reality, the author was the late Christopher Hitchens.) As you might expect, people who identified as Republicans saw the review as more neutral, regardless of who they thought wrote it. The strange thing is that strong Democrats actually saw the review as slightly in favor of Democrats when they believed it was written by a Democrat! We interpret criticism completely differently depending on how we see the relationship between ourselves and the author.

What’s a journalist to do?

The first defense against accusations of bias is to report fairly. But the hostile media effect pretty much guarantees that some stories are going to be hated by just about everyone, no matter how they’re written. I suppose this is no surprise for any journalist who reads the comments section, but it has implications for how news organizations might respond to such accusations.

This research also suggests that the longstanding practice of journalists hiding their personal affiliations might actually be effective at reducing perceived bias. But only up to a point: To avoid charges of bias, the audience needs to be able to see the journalist as fundamentally one of them. This might require getting closer to the audience, not hiding from them. If we each live inside of many identities, then there are many possible ways to connect; conversely, it would be helpful to know, empirically, under what conditions a journalist’s politics are actually going to be a problem for readers, and for which readers.

We might also want to consider our framing more carefully. Because perceptions of bias depend on how we are thinking about our identity in that moment, if we can find a way to tell our stories outside of partisan frames, we might also reduce feelings of unfairness. The trick would be to shy away from invoking divisive identities, preferring frames that allow members of a polarized audience to see themselves as part of the same group. (In this regard, the classic “balanced” article that quotes starkly opposing sides might be a particularly bad choice.)

Encouraging the audience to perceive itself as unified — this seems simplistic, or naïve. But the consideration of identity is foundational to fields like mediation and conflict resolution. Experimental evidence suggests that it might be important in journalism too.