Some Fact Checkers Are Better Than Others

Since I've become a pretty outspoken critic of "fact checking", I often get asked if there's any media fact checking efforts I approve of. The short answer is no, not really.

The long answer is that, while they're all bad, some are better than others. The Center for Media and Public Affairs at George Mason University, whose work studying the results of media fact checking I have previously noted, is out with a new analysis. They've compared the work of PolitiFact to the Washington Post's fact checker, Glenn Kessler:

PolitiFact rated Democratic claims as “mostly true” or “entirely true” about twice as often as Republican statements -- 42% true ratings for Democrats vs. 20% for Republicans.

Conversely, claims by Republicans were rated as entirely false about twice as often as Democratic claims – 29% false ratings for GOP statements vs. 15% false ratings for Democrats. (This includes categories labeled “false” and “pants on fire.”)

By contrast, on a scale of zero to four Pinocchios, the Washington Post Fact Checker rated Democrats as more likely than Republicans to make both the most truthful and the least truthful claims:

26% of Democratic claims were awarded zero or one Pinocchio (the most truthful categories), compared to 16% of Republican claims. But 61% of Democratic statements also received three or four Pinocchios (the least truthful categories), compared to 48% of Republican statements.

Overall, Democratic claims averaged 2.52. Pinocchios, almost identical to the 2.48 average for GOP claims analyzed by the Washington Post Fact Checker. At Politifact.com. On a scale from 1 (true) to 6 (pants on fire), Republicans averaged 2.56, compared to 1.77 for Democrats.

PolitiFact's bias is quite overt, and worse than that, they're uninterested in explaining why they call Republicans liars at two the rate of Democrats. (They also have a lot of explaining to do regarding their abuse of facts in specific rulings.)

By contrast, I'm not surprised to see that Washington Post fact checker Glenn Kessler appears to be more even-handed. In my experience, Kessler is generally more reasonable. For instance, when I criticized one of his rulings he linked to my critique to offer readers another perspective. He openly admits that rating statements by "pinnochios" is a gimmick, and he's also the only fact checker to acknowledge that Romney has valid reasons for saying the Obama administration undermined welfare reform's work requirements. Other fact checking organizations botched the issue in spectacular fashion.

However, I've found some of Kessler's individual rulings appalling, and he's clearing a very low bar relative to PolitiFact. Kessler deserves credit for making more of an effort, but your best option is still using your own discerning judgement to check your own facts—particularly if you're among the half of America who's right-of-center views get the short shrift by journalists.