When Facts Aren’t Enough: Rethinking the Role of Fact-Checking

Cory Barker: In December 2016, a man walked into a Washington, DC, pizza shop carrying an assault rifle. He wasn't there for lunch. He was there to self-investigate a conspiracy theory he'd found online, one claiming that the restaurant was at the center of a secret child trafficking ring linked to high-ranking politicians. Just one problem, it wasn't true. Not even close. There was no basement. There was no ring. Just a viral lie that spiraled out of control. And even after it was thoroughly debunked, some people still believed it. That story, which became known as Pizzagate, became a wake-up call for the power and danger of misinformation. But it also raised a bigger question. What do we do once a falsehood takes hold? And when the truth finally shows up, does it even matter? That's where fact checking comes in. It's supposed to be the fix, an evidence-based response to false claims. But in a media landscape full of polarization and distrust, fact checking isn't always seen as neutral or even helpful.

Matt Jordan: On this episode of News Over Noise, we're digging into the role of fact checking in the news and information ecosystems, how it works, when it works, and why it sometimes backfires. And to help us make sense of that, we're talking with Dr. Michelle Amazeen, the associate dean of research and associate professor of mass communication at Boston University. Michelle also runs the Communication Research Center, where she studies persuasion, misinformation, and how people engage with media corrections. She's one of 22 global scholars behind The Debunking Handbook 2020, has advised everyone from the FTC to the EPA, and her upcoming book Content Confusion, digs deep into how native advertising blurs the lines between news and influence. Michelle Amazeen, welcome to News Over Noise.

Michelle Amazeen: Thank you for having me.

Matt Jordan: Can you tell us a little bit about the history of fact checking as a form of research and media content?

Michelle Amazeen: Fact checking as a form of research, I would say, started around the 20 late aughts into around 2010, following the emergence of factcheck.org, politifact.com. Although there was Brendan Nyhan's Spinsanity, I think that was 2001. But that was a very short-lived effort before Brendan, oh, went back into research and started studying misinformation and political science. So, I think we could say as it relates to fact checking, that is about when the research end of it [INDISTINCT], although I'll amend that slightly and say that there was some research done regarding ad watching following the 1988 presidential election between Dukakis and Bush. So, into the 1990s, there was some research on these ad watching segments on broadcast news that followed the 1988 presidential election, which was perceived largely as very nasty, negative, with lots of factual inaccuracies.

Matt Jordan: So, it's really a response to a kind of bad faith communication or the emergence of that as a political strategy? Is that your thought about this?

Michelle Amazeen: I would say so, although we could actually extend this back even further, 100 years earlier, with the muckrakers of the Progressive Era, who were holding officials to account for all the horrific conditions that was happening with industrialization and the rampant disinformation that was circulating with the patent medicine producers. I think there was—some of the muckraking journalists led to some of the legislation that was put into place around that time period based upon the meatpacking conditions. And I think it was the Pure Food and Drug Act. I forgot what year it was precisely. But there's an argument, which I have made, that that was an early form of fact checking.

Cory Barker: So how has fact checking evolved in American journalism since that period? You just went back pretty far. So, as we move further into the 20th and now into the 21st century, what has sort of changed most about fact checking as a process in that time?

Michelle Amazeen: Well, I would say that what has changed since the emergence of factcheck.org in 2004 is the institutionalization of fact checking. So now, there's this critical mass of fact checkers that may have at one point seen themselves as in competition with one another. Now, they are working with one another to try and bring standardization to processes in how fact checking is conducted. There is an independent group called the Independent Fact-Checking Network, the IFCN, which has essentially institutionalized fact checking around the world.

Matt Jordan: What have they instituted that has spread around the world?

Michelle Amazeen: Yeah. So, I would say what they've institutionalized is the process that practitioners, fact checking practitioners, follow in order to be considered an IFCN signatory, a signatory of their principles. So, they have a set of principles that have to do with being transparent about your sources, about being fair, and so forth.

Cory Barker: What's really different about these forms of public facing fact checking versus the fact checking processes that journalists and folks who worked in newsrooms have done for years, folks who work for magazines, who are following up with sources, who are in stories? Are there differences? Or is it really just public versus private practice?

Michelle Amazeen: Yeah, that's a great question. There is a difference. So, what you've articulated there is a difference that I have called internal versus external fact checking.

So the internal fact checking, is what you're referring to, has been a century worth of practice where you have internal fact checkers at a magazine or at a newspaper who goes through and makes sure that the quote is right before the content is published, whereas external fact checking is the process of determining whether the information is accurate or not publicly.

So, it's verifying already circulating claims to determine the accuracy of those claims. Is it true? Is it mostly true? Is it misleading? And why?

Matt Jordan: Do you see any problem with the externalization of this as an institutional practice in the news ecosystem? Is it a problem that pits, say, a lot of political journalism as access-oriented? Is this something that news orgs would rather have externalized so that they can preserve those contacts and not say, hey, we're not the ones pushing back on your outrageous claims here, we're just printing things as you see them? Or is there a conflict of interest that is established or a problem in the news system that is established when this all gets externalized like this?

Michelle Amazeen: Well, Bill Adair of politifact.com would argue and has argued that he got into the practice of external fact checking because he felt so guilty about when he was in journalism and the journalism industry's failure to do just that, failure to hold officials to account for their claims and whether they were true or not, the position being that that is what journalism is supposed to do. Otherwise, essentially, you're dealing with glorified stenographers.

Matt Jordan: So, one thought about that is essentially what fact checkers are doing is they're looking at veracity. And a lot of what we know, the media cycle and the news cycle, about is agenda setting. Is there an issue with the only thing that people are worried about or thinking about in terms of the way journalism works, is just whether or not you're verifiably true? Or is there something—does all of the displacement about political journalism get put on whether what he says is true, not about what he's talking about or the way that the agendas get set?

Michelle Amazeen: Yeah, so that's an interesting question. Journalism is more than just whether the claim is true or not. Of course, there's helping your readers understand why they're saying what they're saying. So, Steven Lewandowsky is a researcher in the UK. And he's recently coined the term truth seeking versus belief speaking. And this kind of has to do with the way individuals process information. So, there's some people who are looking for the truth versus other people who are more about, how does it feel? How does the information feel? Whether or not it's accurate or not, it feels right. And you want it to be right. So, people who have the truth-seeking mentality are more receptive to fact checkers, whereas the belief speakers are more adversarial.

Matt Jordan: Is that a cultivated disposition to a certain degree? Which is to say, you're describing fact checking as emerging at a moment when we see the emergence of partisan and partisan media. And those are audiences that are cultivated to be the believer types of the ones that you described. So, do we see people starting to attack fact checkers in order to preserve that kind of audience disposition, say, the early aughts? I guess I ask that because I know from the research that sometimes, when your fact check something for a group that is an audience of believers, it ends up reinforcing the belief. It's like as if it has been fact checked. They believe it more in a way.

Michelle Amazeen: Yeah, so you're getting into the defensive processing and the motivated reasoning. I think that there's some evidence of that. But I think that there's also—audiences aren't monolithic. So, you have some people who process information for accuracy. They want to know whether something's accurate or not before they make decisions. And then there's other people who are more dispositionally oriented to processing information defensively and trying to either preserve their worldview or their identity and act accordingly.

Cory Barker: What would you say the typical news reader might misunderstand about the processes of fact checking?

Michelle Amazeen: So, I've heard in some of my research, when I've talked to media consumers about fact checking, many of them aren't familiar with it. They're unaware. And I think there's an issue of credibility. Like, who are these people? How do we know that they don't have an ax to grind? So, they're unaware about the international fact-checking network and the set of priorities that they have for being independent and fair and transparent, that they're not privy to in understanding what fact checking is. And is that a failure of fact checkers for not communicating that as much as they should be? I think that's a fair question.

Cory Barker: Yeah, I mean, some of this feels like just a repetitive kind of self-generating ecosystem, where if you are someone who's maybe distrustful of certain publications or certain institutions in general, it's sort of difficult, at least for me, to imagine someone like that being persuaded by a fact checker at The Washington Post if they don't really trust The Washington Post or, quote unquote, "elite media" in general.

Michelle Amazeen: Right.

Matt Jordan: So, fact checking has been in the news a lot recently. So, Meta had this enormous network of fact checkers that they had built up over time—and the group that you described, the international group, was part of that—and just announced that they're not going to do fact checking and instead are going to do community notes. What's your feeling about why they did that? Was this because of the people that don't like it? Or is it people don't like interacting with media where they feel there are fact checkers lurking about? Or is there something else involved?

Michelle Amazeen: Yeah, so I wondered the same thing. I think it's merely political posturing. I mean, Mark Zuckerberg went on Joe Rogan right after he did that and stated that most people don't like fact checking. It's become problematic, blah, blah, blah. The evidence, however, suggests otherwise. So shortly after this announcement, my university, Boston University, we fielded a nationally representative study shortly after he made that announcement. So, this was mid-January of this year when we did this study. And what we found is that Americans do support content moderation on social media sites, particularly in the realm of health topics. And specifically, we found that 63% of our participants were in favor or agreed that independent fact checkers are a good thing to have on social media. And we also asked them about the community notes, which is where Meta is pivoting using this community notes process. Only 48% of Americans agreed that the community notes was something that was a good idea. So overwhelmingly, Americans are supportive of content moderation. And they prefer independent fact checkers to do that over community notes.

Matt Jordan: It seems that most Americans then are right because what we know about community notes is that they just aren't very effective, not only at flagging content. They're easy to manipulate. And most people blow right by them. Like, on X, they're in a nice powder blue. That kind of almost draws attention to something other than the falsity of the information. So, they're just not effective.

Michelle Amazeen: Well, the academic evidence I've seen is mixed. There are times when it could be effective under ideal circumstances. However, how long has X had their community notes program, right? Well, over a year? 18 months? Something like that? What is the state of conversation on X? I mean, there's a lot of disinformation. There's a lot of hate speech. It's not a great place to be suggesting testing that the community notes has not been effective in practice.

Cory Barker: And it seems like, to some extent, the decision to shift more towards community notes is obviously part of a much longer history of these social platforms trying or at least publicly suggesting they're going to try to attack or combat misinformation and maybe not being fully invested in what that looks like. Or I know in some of your writing, you talked about how the folks who did work as fact checkers for Meta weren't always totally sure what they were doing or who they were reporting to or what the process was like. I mean, is the real issue that just relying on the structure of these companies at all to provide any sort of verification or fact checking or assuming that they're going to do anything that we might consider as right in relationship to misinformation?

Michelle Amazeen: Well, that's an interesting question. I mean, the way it is now is these social media platforms are shirking their responsibility for the content on their platforms. Of course, they're shielded by Section 230. But this community notes is just another way of pushing the responsibility of maintaining the content on their platforms to the public. So, here's these for-profit companies that are making money hand over fist. And they're actively harming the public.

Matt Jordan: I've been really interested about how fact checking is framed in the media. It got very politicized as opposed to—if you think back to the way you opened by talking about fact checking is emerging out of the Pure Food and Drug times, back when people used to be worried about quack medicine in print and things like that, they wanted to know that they were safe and that the information they were getting is safe. And I think fact checking or content curation as a way of thinking about this on social media is also coming from that same space. So, in the period after a January 6, 2021, we had this kind of Come-to-Jesus moment, where all these platforms said, oh, we're going to do increased fact checking. We're going to do all this extra stuff. And then it got politicized, that the ability to put out disinformation as a strategy was thwarted. And all of a sudden, it was starting to get framed as censorship, as if it was the removal of people's ability to have free speech when actually fact checking is just more speech. It's where you say something. I say, no, it's not. It's actually this. And then that's a dialogue.

But the way that it's been politicized, where in the vice-presidential debate, they agreed to a no fact checking and that was a big point of gotcha moment, you said you weren't going to fact check. All these things that—it has become very politicized. Do you see a way out of that since it's been politicized and fact checkers have fallen into that framing?

Michelle Amazeen: So yeah, there's this argument that fact checking is somehow censorship or censorious. And it's not, as you pointed out. It's just giving more information. In some of the research I've conducted, focus groups talking to people about how they're using media, how they deal with misinformation, and then showing them different interventions, fact checking being one of them, that was some of the concern that came up during some of the groups, that the fact check was censoring the content. There were a few others in the group who corrected them, showing, well, no, look, it's not censoring anything. You can still click on it and see the post. For a while, there was—I think on Instagram, there would be some cover that was put over the post so that it was blurred. And there was a message that this has been found to be inaccurate or whatever it is by fact checkers. And you could click on it and still see the post. But that wasn't clearly understood by everybody. So, the way out of this, I think, is continuing to help people understand what fact checking is, what it is not. It is not censorship. Having legislators understand this and helping constituents understand this—I think we do need to revisit section 230 and how that's shielding these social media companies. Yeah, so just additional conversations about this. And I think revisiting some of the legislation that we have that's protecting social media.

Matt Jordan: So, fact checking has also emerged as a feature of online interaction as well. And some of the way that it's presented to the public has been kind of shaped by that. And I'm thinking of PolitiFact's meter. They have this kind of graphic. And it goes up to pants on fire. Do you think that that contributes to some of the antipathy toward it? Is it shaming too much for those who are—is there too much there?

Michelle Amazeen: Yeah. So, I've actually looked at this. And two things come to mind about this. First, we've actually tested to see whether fact checkers that use some sort of ratings meter like that versus those that don't—like, factcheck.org doesn't have that. They just use contextual corrections, just language. We found no evidence that there was any significant difference in how audience members were affected by those types of fact checks. Now, some of the fact checkers who used those ratings meters, like Glenn Kessler of The Washington Post, readily admits that those are just gimmicks. There is no scientific difference between two Pinocchios and three Pinocchios or between half true and mostly false. It's just a shorthand for audience members to understand the degree of falsity or inaccuracy. In a separate study, we asked the public, which type of fact checking they preferred, ones that fact checks that used a ratings meter versus ones that didn't. And it was essentially split, half and half. Some people prefer the shorthand graphics. Other people just want the context.

And I think this goes back to how people are different. And we have different ways of processing information, different preferences in how we process information. Some of us are visual learners and therefore might prefer the visual graphics. Others of us are not and don't prefer that.

Cory Barker: To follow up on that a little bit, it seems to me that there's a bit of a disconnect between people saying in surveys or in conversations in your research that they want fact checking, whether that's from social platforms, news organizations, et cetera. But we see obviously a disinterest in potentially subscribing to news organizations or paying for news or, as Matt mentioned, just scanning over community notes or any sort of fact checking happening on social platforms. So, I'm just wondering what you think news organizations specifically could do to make their fact checking efforts more legible or available to people who say they want this but maybe they're not interested in certainly paying for news or maybe even visiting news websites. I mean, obviously, you just mentioned, everybody's different. Some people respond to graphics. Some don't. But are there things that you think news organizations can do to get this type of stuff out in front of a public who is saying they actually want it but maybe are not going to find it themselves?

Michelle Amazeen: I do think that news organizations could do a better job incorporating the work of—especially if it's their own fact checkers. Like, at The Washington Post has the Fact Checker, Glenn Kessler's team. Is that really interwoven within all of their other reporting? I'm not sure how much it is. So, I think more integration throughout the newsroom and the reporting that's coming out in the newsroom. There have been times when I've seen an article from a news outlet where they had fact checked something. And then there's this separate article that repeats the inaccuracy that the fact check had corrected. So clearly, more communication within the newsroom and then also promoting that they are checking the accuracy of information so that you, as the reader, don't have to and showing you how we came to our conclusions.

Cory Barker: Yeah, that's going back to something we were talking about earlier. It sort of feels like the situation you just described, with a division between public facing, fact checking, and the newsroom as a whole, is in part to avoid the politicization question, that we have this thing over here that is doing fact checking. But that process or that individual person is often accused of censorship or being, quote unquote, "political." And we want to keep that away from the newsroom. And yet, that leads to these situations where maybe people are not seeing it because the fact checking operation or desk has its own social media profile. So, it's not integrated into The Washington Post on Twitter or any social platform. And therefore, there's an inherently limited reach of that process, where it's maybe only getting to the people who, as you said earlier, are predisposed to either trust these things or seek out that information, whereas a more general populace who is saying, we want this, but they're not necessarily seeking out or maybe not seeing it to the same level.

Michelle Amazeen: Absolutely. And I would say it's the minority of newsrooms that have their own fact checking initiative. So, The Washington Post is unique in that way. Many of the fact checkers are independent of news organizations. So, I might suggest that there is opportunity for fact checkers to band together, perhaps as part of the International Fact-Checking Network, and just promoting themselves more. One of the things we asked in that poll that I mentioned in January, when we were asking about how people felt about fact checking, we asked people how willing they would be to donate $1 to fact checkers. And we found that a third of our participants in the US, US adults—a third of them said that they would agree that they'd be willing to do that. So that may sound like a small number, a third. But yet, that could be real money. So, crowdfunding of fact checkers. And with social media platforms, like Meta, pivoting away from fact checkers and the fact checkers are thereby losing that financial support that meta used to provide, they need the funding now more than ever. And here's some evidence that some Americans, third of them, might be willing to chip in a bucket. Kind think of it as support people give to our public broadcasters, like NPR, PBS, which the government is now trying to do away with as well. But $1 a year—you can't even get a cup of coffee for $1.

Matt Jordan: Yeah. No, it's interesting. And as our news system moves away from a subscription model, where you pay to get it to a kind of a aggregated social media model that is more of an attention economy, maybe it's that there's less interest in individual news orgs to make their brand about trustworthiness. People talk and there's a lot of description of the environment we're living in now as being post-truth or whatever one wants to call it. But in an attention economy, there's a liar's dividend. It doesn't hurt you if you print false things or if you say false things if it is just about clicks. And it seems like people want it. But at the same time, that's not the way the economic system is set up so that people aren't going, as we tell them all the time on our podcast, to go to a trusted news source. They're kind of seeing what comes their way. What's the algorithm going to serve up to it? What's going to engage me? And that kind of lack of concern about the brand of a news org as being trustworthy or not is pretty telling.

Michelle Amazeen: Absolutely. So, you've identified a structural challenge for media consumers now. We're in an era, like you said, where everybody's getting things online frequently on social media. Across numerous studies I've done, I've seen that more than half the time people don't even realize what the source is of whatever article they've just shared with their friend. They're not looking at the source. So, as you said, this flattens out trustworthiness. Does it matter that it's from The Washington Post as opposed to Joe's blog? So, it's the structure of our media system that rewards sensationalism, negativity. And this is the environment in which legacy news outlets are trying to compete and finding that their legacy branding is mattering less and less. Now, they're competing with influencers.

Matt Jordan: Who aren't very fact checkable.

Michelle Amazeen: And let me add something on this issue of promotion. Again, I think that the fact checkers could be and should be doing a better job of promoting themselves, whether they're within a news organization or without standalone. We see there's examples in the Trump administration right now. So, Kristi Noem, the secretary Department of Homeland Security, she's got this massive advertising campaign going on right now, where she's imploring immigrants to self-deport. I've seen this numerous times on television. Yes, I'm Gen X. And I watch television news. But I think the AP reported that that's $200 million that has been paid for this advertising campaign. That's our tax dollars that's paying for that. And it's just not nationally. I live in Massachusetts. And our secretary of state, William Galvin, he also has been employing television commercials to—it's like a mix of public service but also promoting, we see this guy. Oh, who is this guy? Oh, he's helping us. He's telling us about this. And there's even been—I remember hearing about—this was in North Carolina. The labor commissioner in North Carolina, Cherie Berry, she was the one who signed all of the inspection reports for the annual elevator inspections. How many elevators are in a city or in a state? Well, in every elevator in North Carolina, there was that inspection certificate with a photograph of Cherie Berry. So not just her signature but her photograph. And there was evidence that that helped her election—name recognition. And they recognized, oh, yeah, that's the lady from the elevator. So, in the same way, the fact checkers have been very quiet about what they do and what benefits they bring. So, I think we need to see more self-promotion of fact checkers.

Matt Jordan: CNN has—what is his name? Oliver Darcy—or is it?

Michelle Amazeen: —anymore.

Cory Barker: He got laid off.

Matt Jordan: He got laid off?

Cory Barker: Yeah.

Matt Jordan: Well, I was going to say—

Michelle Amazeen: Something though.

Matt Jordan: —because he did have a kind of almost branded persona. They would bring him on after a debate. And he would say, here are the 20 things that were absolutely outrageously not true. And it was kind of a televisual thing that you could see why this worked as a business plan. But you know, then—

Michelle Amazeen: Back to your point, Matt, earlier about how that might have impeded some of the access that CNN was able to get with this new administration—get rid of that guy. He's pointing out too many inconvenient things.

Matt Jordan: Where do you see the future of fact checking moving to? I mean, is this something that AI is going to fix for us. I'm sure the entrepreneurs in that space would like to make that claim. Where do you see it moving?

Michelle Amazeen: I think AI is a tool that will facilitate fact checking. And I know that fact checkers are already playing with AI to facilitate their fact checking efforts. Research has shown that generally disinformation repeats on the topic of vaccines or climate. There's common fallacies that are told. Same with election lies. You'll see them repeat from one country into a different country. So being able to predict what those types of claims might be, they're using AI to help find them and correct them. I think that another area that some fact checkers are starting to leverage is from an education standpoint. So, fact checking is a reactive process. The disinformation, the inaccuracy’s already out there. And we know from research that it's harder to correct a fact once it's out there. So, some fact checkers are turning to preemptive corrections or pre-bunking, leveraging inoculation theory, forewarning people about, here's the types of things that we may hear. And that's tied in with media literacy. So, helping the public to become more media literate. I know politifact.com is owned by Poynter. And they have their MediaWise partnership. That's part of Poynter as well. And MediaWise is all about media literacy and helping people understand more about how disinformation travels. And so they've put together different types of education programs targeted at high school students, senior citizens, Spanish speakers, and so forth.

Matt Jordan: I think you're right that such a large degree of the misinformation that we see is actually kind of what we would call sticky narratives, things that come up again and again and again. Globalists theories of puppet masters behind this—these are things that we've seen for hundreds of years. So, is there evidence that inoculation works in those cases?

Michelle Amazeen: Absolutely, yeah. So, I've done some research on this. Others have done research on this. Inoculation is very effective at helping people become more resilient to disinformation. I've done some research on how it works in tandem with fact checks. And so, I think what we found is that certain people are more receptive to fact checks. As we talked about previously, others don't like fact checking. But those same people may be more receptive to these inoculation, these media literacy pre-bunking. They think that that is more acceptable than fact checking.

Matt Jordan: Well, Michelle, thank you so much for helping us to understand this really important topic and share a little bit of your research with us. And we look forward to reading more of your stuff.

Michelle Amazeen: Thank you so much.

Matt Jordan: Cory, we heard a lot of really interesting stuff there. What are you going to take with you the next time you open the news?

Cory Barker: I think Michelle made a lot of great points about the disconnect between what people say they want related to fact checking and how various fact checking organizations or newsrooms actually bring those fact checks to the news consumer. And I think it gets us back to this idea of how do news organizations or people generally involved in news and information reach the contemporary public when they have so many other types of information and content creators, like pulling at their eyeballs and attention. And that raises a lot of concerns and questions about how we get quality, verifiable information even when we say we want it.

Matt Jordan: Yeah, no, I'm always interested to hear about an institution or something that's been institutionalized within the media ecosystem, like springing up. Like, it must be responding to something. And I think it's responding to the lack of awareness that people have about what journalistic ethics are because they're so used to interacting with social media, where there are none. And so, you have this institution that has to emerge to fill that gap. So again, my takeaway from this is it's more important than ever that we know the difference between journalists that follow verification and those who don't. That's it for this episode of News Over Noise. Our guest was Dr. Michelle Amazeen, associate dean of research and associate professor of mass communication at Boston University's College of Communication. To learn more, visit newsovernoise.org. I'm Matt Jordan.

Cory Barker: And I'm Cory Barker.

Matt Jordan: Until next time, stay well and well informed. News Over Noise is produced by the Penn State Donald P. Bellisario College of Communications and WPSU. This program has been funded by the Office of the Executive Vice President and Provost at Penn State and is part of the Penn State News Literacy Initiative.

[END OF TRANSCRIPT]