Lil White Lies Extra: Do The Hustle!

Well, around we go again. Let me start with the usual explanation. My conservative counterpart (based on this recent post) doesn’t appear to have heard/listened to/understood Eric Levine at his blog Content In Reality with whom he had a lengthy comment exchange early this year (although it certainly does not surprise me). As Levine carefully explained in regard to the allegation of selection bias by PolitiFact, PolitiFact Bias creates a straw man (note: the “article” referred to in the quote is from a PolitiFact Bias’ page where selection bias is the first issue noted warning readers not to trust PolitiFact):  (emphasis added)

To give them the most credit, it is possible there may be some liberal selection bias at work here, but not in the way they think. Politifact often gets ideas for statements to check from their readers. It is possible most of their readers are liberal, which sounds quite plausible given the fact that conservatives seem so ready to accuse Politifact of bias. This is no fault of Politifact, nor does it show any bias on their part. The fault lies squarely with Republicans who chose not to submit statements to Politifact and instead waste their time complaining about perceived bias (a vicious cycle). It should also be noted that the definition of Selection Bias is "A type of bias caused by choosing non-random data for statistical analysis." Well duh! The article itself mentions that Politifact Bill Adair noted "We choose to check things we are curious about. If we look at something and we think that an elected official or talk show host is wrong, then we will fact-check it." That right there should show that there is selection bias, though not liberal selection bias. However, the article intends to show that the bias is toward the left, instead of toward important statements. If the article was intended to merely accuse Politifact of selection bias, they would basically be attacking a straw man since the main goal of Politifact is not to tally true and false statements to figure out which candidates are more honest…

Levine also refers to the selection bias that writer Bryan White accuses PolitiFact of as “party-related” selection bias, while White makes no distinction. This is very important because Levine notes that in order to prove party-related selection bias, it would have to be shown that the entire crew of PolitiFact, including the three editors who “vote” in agreement on the Truth-o-Meter ruling, are liberal. On top of that, other fact-checkers, such as Washington Post’s FactChecker and FactCheck.Org, who may have made the same fact-checks and reached the same conclusions, would have to be liberal in the same way as well. So we have a good reason why White prefers to use the generic “selection bias” without informing his readers that there are different types, and what evidence is required to prove the type (party-related) he contends PolitiFact has. 

Keeping that in mind, it appears White may be using selection bias as a way to influence his readers to believe there’s a false choice (i.e., selection bias equates to only party-related selection bias for which PF is at fault for having no solution) because there is no way, as far as I know, for PolitiFact to “control” for such selection bias. As PolitiFact Editor in Chief Bill Adair has stated:  

It's fine if people want to use our report cards to compare the records of people we fact-check. But it's important to be accurate about what those report cards say and what they don't," Adair said. "The report cards provide a tally of the claims we chose to check. But it's not accurate to say the report cards indicate who ‘told the most lies.’ We are journalists who choose our fact-checks based on what is newsworthy. We are not social scientists and are not using any kind of random sample to select statements to check.

So, it’s been again re-hashed here as an intro for a post in response to White’s allegations of “truth hustling” by Chris Mooney, who wrote an article in something called the Desmogblog called “More Evidence that Republicans are more factually challenged” in which he re-verified an analysis of every fact-check by Glenn Kessler in the Washington Post Fact Checker, with a conclusion (somewhat similar to what I’ve done with the Truth Index) that based on the fact checks, Republicans tell more lies than Democrats, and he concludes with a query—“there has to be an explanation”(so, what is it?):

In my opinion, what they're telling us is that Republicans today are simply more untruthful than Democrats—and the difference is dramatic enough that even a centrist-leaning fact checker, who strives to be non-partisan, cannot fail to pick it up.

My conservative counterpart, of course, would have none of it; he presents his false dichotomy thusly:

If Kessler and the PolitiFact folks thought of themselves as partisan, maybe it would increase the chances they would institute controls on selection bias. Mooney's self-confessed reassurance is misplaced. In truth, the self-perception of bias is irrelevant. A properly done study controls for bias so that the point of view of the researcher is irrelevant. But Kessler and PolitiFact apparently do absolutely nothing to control for selection bias.

Bill Adair states above that he’s not a social scientist. White knows that “controls on selection bias” is an impossibility. But because he infers that the only selection bias “that exists” is party-related, the only conclusion that can be reached is that both Kessler and PolitiFact (and my reviews of FactCheck.Org indicate the same), until they institute such controls, are all suffering from liberal (party-related) selection bias, and therefore should not be trusted.

White’s conclusion, however, dances around with distortion and catches his pants on fire, to Mooney’s original query of why there has to be an explanation of why Democrats score as more truthful, emphasis added, link mine.

The answer to the question isn't that tough. The data are not designed to give us a dependable differential breakdown for the claims of Democrats against those of Republicans. No serious researcher would try to answer that question while relying on left-leaning journalists with no control for selection bias. But the data do have an obvious use. The data directly measure how fact checkers choose subject matter. Kessler and PolitiFact give us an excellent tool measuring their selection bias. Add to that the ratings each fact checker provides and we end up with data that can help us measure media bias.

I didn’t know he knew that everyone who worked at PolitiFact National, as well as Glenn Kessler, were “left-leaning journalists”. Where’s his evidence? Does he have copies of their voting records and political donations? Did any one of them work for Bernie Sanders or write for Current TV or even one of the MSNBC evening talk shows? If he thinks his targeting (mostly) Lou Jacobson through his “Grading PolitiFact” reviews is sufficient evidence of “left leaning”, he better think again. Weak, anecdotal write-ups by someone who’s admitted their own bias do almost nothing to prove PolitiFact journalists are liberal-leaning.

The data having an “obvious use” for measuring selection bias is a, pardon me, stupid answer. It is NOT obvious. Again, he ignored everything Eric Levine explained to him. White’s assuming that this is only a measure for liberal selection bias, when there are many, many reasons why the data might show Republican make more statements found less truthful than Democrats. As Levine also noted, the data was evenly divided in 2007 and calculations on that data at that time showed the Republicans ahead Truth Index-wise with fewer Pants on Fire than the Democrats(see graphic below). So if the PolitiFact writers were in fact liberally biased, what were they doing then? Did they all change their minds and go from leaning right to leaning left? You see how ridiculous he sounds.

Click to enlarge:  Eric Levine's capture of 2007 PF ruling data show at that time the Truth Index favored Republicans.

 Finally, there’s the “control for selection bias.” If White was worth his salt, he would publish something, anything that would tell readers what that control might be. As far as I know, while he’s complained prolifically about selection bias, he’s never proposed an alternative. If he expects PolitiFact to select statements randomly, he already has an answer from Bill Adair—they are not social scientists. They’ve published that before, so White should know about it. I’d predict his response to this might be that the burden should be put on PolitiFact; but White is the one making the complaint of bias. So if there’s another option, he should suggest it. If not, he should, well, I’ll just come out and acronym it, STFU.