Well, around we go again. Let me start with the usual explanation.
My conservative counterpart (based on this
recent post) doesn’t appear to have heard/listened to/understood Eric Levine at his blog
Content In Reality with whom he had a lengthy comment exchange early this year (although it certainly does not surprise me). As
Levine carefully explained in regard to the allegation of selection bias by PolitiFact,
PolitiFact Bias creates a straw man (note: the “article” referred to in the quote is from a PolitiFact Bias’ page where selection bias is the first issue noted warning readers not to trust PolitiFact): (emphasis added)
To give them the most credit, it is possible there may be some liberal selection bias at work here, but not in the way they think. Politifact often gets ideas for statements to check from their readers. It is possible most of their readers are liberal, which sounds quite plausible given the fact that conservatives seem so ready to accuse Politifact of bias. This is no fault of Politifact, nor does it show any bias on their part. The fault lies squarely with Republicans who chose not to submit statements to Politifact and instead waste their time complaining about perceived bias (a vicious cycle). It should also be noted that the definition of Selection Bias is "A type of bias caused by choosing non-random data for statistical analysis." Well duh! The article itself mentions that Politifact Bill Adair noted "We choose to check things we are curious about. If we look at something and we think that an elected official or talk show host is wrong, then we will fact-check it." That right there should show that there is selection bias, though not liberal selection bias. However, the article intends to show that the bias is toward the left, instead of toward important statements. If the article was intended to merely accuse Politifact of selection bias, they would basically be attacking a straw man since the main goal of Politifact is not to tally true and false statements to figure out which candidates are more honest…
Levine also refers to the selection bias that writer Bryan White accuses PolitiFact of as “party-related” selection bias, while White makes no distinction. This is very important because Levine notes that in order to prove party-related selection bias, it would have to be shown that the
entire crew of PolitiFact, including the three editors who “vote” in agreement on the Truth-o-Meter ruling, are liberal. On top of that, other fact-checkers, such as Washington Post’s FactChecker and
FactCheck.Org, who may have made the same fact-checks and reached the same conclusions, would have to be liberal in the same way as well. So we have a good reason why White prefers to use the generic “selection bias” without informing his readers that there are different types, and what evidence is required to prove the type (party-related) he contends PolitiFact has.
Keeping that in mind, it appears White may be using selection bias as a way to influence his readers to believe there’s a false choice (i.e., selection bias equates to
only party-related selection bias for which PF is at fault for having no solution) because there is no way, as far as I know, for PolitiFact to “control” for such selection bias. As PolitiFact Editor in Chief Bill Adair
has stated:
It's fine if people want to use our report cards to compare the records of people we fact-check. But it's important to be accurate about what those report cards say and what they don't," Adair said. "The report cards provide a tally of the claims we chose to check. But it's not accurate to say the report cards indicate who ‘told the most lies.’ We are journalists who choose our fact-checks based on what is newsworthy. We are not social scientists and are not using any kind of random sample to select statements to check.
So, it’s been again re-hashed here as an intro for a post in response to White’s allegations of “truth hustling” by Chris Mooney, who wrote an article in something called the
Desmogblog called
“More Evidence that Republicans are more factually challenged” in which he re-verified an analysis of every fact-check by Glenn Kessler in the
Washington Post Fact Checker, with a conclusion (somewhat similar to what I’ve done with the
Truth Index) that based on the fact checks, Republicans tell more lies than Democrats, and he concludes with a query—“there
has to be an explanation”(so, what is it?):
In my opinion, what they're telling us is that Republicans today are simply more untruthful than Democrats—and the difference is dramatic enough that even a centrist-leaning fact checker, who strives to be non-partisan, cannot fail to pick it up.
My conservative counterpart, of course, would have none of it; he presents his false dichotomy thusly:
If Kessler and the PolitiFact folks thought of themselves as partisan, maybe it would increase the chances they would institute controls on selection bias. Mooney's self-confessed reassurance is misplaced. In truth, the self-perception of bias is irrelevant. A properly done study controls for bias so that the point of view of the researcher is irrelevant. But Kessler and PolitiFact apparently do absolutely nothing to control for selection bias.
Bill Adair states above that he’s not a social scientist. White knows that “controls on selection bias” is an impossibility. But because he infers that the only selection bias “that exists” is party-related, the only conclusion that can be reached is that both Kessler and PolitiFact (and
my reviews of FactCheck.Org indicate the same), until they institute such controls, are
all suffering from liberal (party-related) selection bias, and therefore should not be trusted.
White’s conclusion, however, dances around with distortion and catches his pants on fire, to Mooney’s original query of why there has to be an explanation of why Democrats score as more truthful, emphasis added, link mine.
The answer to the question isn't that tough. The data are not designed to give us a dependable differential breakdown for the claims of Democrats against those of Republicans.
No serious researcher would try to answer that question while relying on left-leaning journalists with no control for selection bias. But the data do have an obvious use. The data directly measure how fact checkers choose subject matter. Kessler and PolitiFact give us an excellent tool measuring their selection bias. Add to that the ratings each fact checker provides and we end up with data that can help us measure
media bias.
I didn’t know he knew that everyone who worked at PolitiFact National, as well as Glenn Kessler, were “left-leaning journalists”. Where’s his evidence? Does he have copies of their voting records and political donations? Did any one of them work for Bernie Sanders or write for Current TV or even one of the MSNBC evening talk shows? If he thinks his targeting (mostly) Lou Jacobson through his “Grading PolitiFact” reviews is sufficient evidence of “left leaning”, he better think again. Weak, anecdotal write-ups by someone who’s admitted their own bias do almost nothing to prove PolitiFact journalists are liberal-leaning.