Grading PolitiFact *Liberal*-Style: Yes, Scott Walker is #1 "Liar"

A PolitiFact (PF) New Jersey fact-check on Loretta Weinberg, the state senate majority leader, done on June 7, seems to have slipped through my conservative counterpart’s bias-searching fingers, because I’m sure if he had read it he would be quoting it (besides having a major cow!) liberally throughout both his anti-PolitiFact blogs. For Bill Adair, the head of PolitiFact, came right out and directly addressed his biggest complaint of why he despises the Truth-o-Meter concept and what he believes is the reason for its alleged “bias.”

Weinberg said that Scott Walker of Wisconsin topped a list of the  five “Lie - en Governors" according to PolitiFact, and New Jersey Governor Chris Christie was also part of the “top five.” PolitiFact rated this statement “False” and included an interview with PolitiFact Manager Bill Adair, who made this very interesting comment:

"It's fine if people want to use our report cards to compare the records of people we fact-check. But it's important to be accurate about what those report cards say and what they don't," Adair said. "The report cards provide a tally of the claims we chose to check. But it's not accurate to say the report cards indicate who ‘told the most lies.’ We are journalists who choose our fact-checks based on what is newsworthy. We are not social scientists and are not using any kind of random sample to select statements to check."

But did Uppity Wisconsin, the source of Weinberg’s information, have it right? Here’s how PolitiFact explains what they did:

So what is Weinberg talking about?

The state senator’s statement was based on a list posted on the liberal blog Uppity Wisconsin.

The list ranked governors fact-checked at least five times by PolitiFact by the percentage of True and Mostly True ratings they received.

Walker had the lowest combined percentage of those two rulings, according to the June 1 blog post. The post concluded that "Walker is the #1 most dishonest governor in America."

Christie took fourth place among the "top five most dishonest governors in America" with 38 percent of his PolitiFact rulings earning True or Mostly True.

Uppity Wisconsin simply added the percentages of Trues and Mostly Trues, and came up with Scott Walker of Wisconsin at Number One with a combined percentage of 22 percent, followed by Rick Perry of Texas and Lincoln Chafee of Rhode Island both at 25%, Chris Christie of New Jersey at 38%, and lastly Bob McDonnell of Virginia at 44%. Uppity Wisconsin ignored a lot of things in coming up with these numbers. Adair has a point that this does not accurately show “who told the most lies.” For example, in 2012, Rhode Island governor Lincoln Chaffee had only two rulings from PolitiFact. He’s only had eight fact-checks ever from PolitiFact. This is not a good measure of “truthiness” for someone with this few fact-checks.

The more fact-checks on someone, the more it should bolster the claim of measuring honesty. But it’s even hard to say what that “more” number is. I put together a list of those sitting governors with 20 or more fact-checks, and decided "the top five" should be those with 40 or more: governors Chris Christie (NJ), Rick Scott (FL), Rick Perry (TX), John Kasich (OH) and Scott Walker (WI). I threw out Governor Nathan Deal of Georgia (with 22) and Bob McDonnell of Virginia (with 23), simply because I wanted to measure those with the highest number of fact-checks who had been rated recently. Not a lot of politicians meet the “40” criteria, but it’s a better measure than basing it on fewer than 10 fact-checks.

The other problem with Uppity Wisconsin’s measure is it does not take into account the Half Trues, Mostly Falses, Falses and Pants on Fire categories in making its "honesty" measure. The Truth Index as calculated by PolitiFact places heavier weight on the Falses and Pants on Fire categories: the Pants on Fire is weighted 50% higher than a True, because (I’m guessing) it was felt emphasis should be put on the false statements.

It’s just that when you make these grades, show percentages and even have a calculated Truth Index, you risk people running amuck with different calculations of the rulings. Like Jud Lounsbury of Uppity Wisconsin, like Eric Ostermeier, like Eric Levine, and like me.

The top five governors with 40 or more rulings shown by order of Truth Index is graphically depicted below. It lists the biggest “liar” by PolitiFact Truth Index score, which takes into account claims other than True and Mostly True. To the right of the governors’ name in parentheses is his state and the total number of rulings done by PolitiFact, followed by their Truth Index. Rick Perry of Texas has had the most rulings because of his presidential run, which exposed him to PolitiFact National’s ruling scrutiny. His average of negative 23.4 for 122 rulings is about the average for all Republicans (I’ve not published this statistic [yet], but the average for ALL Republican, Libertarian and Tea Party affiliations for rulings by PolitiFact through the end of June, 2012, is negative 23.3) .

Click to enlarge:  Those big orange and red categories on Scott Walker give Loretta Weinberg's claim more credence.

I don’t know if I would call Loretta Weinberg’s claim False.  She's at least partially accurate (Half True).  After all, Scott Walker is the “biggest liar” based on PolitiFact’s own rulings measure, the Truth Index; and Chris Christie is in the list of those with over 40 rulings as well. If she was saying it appropriately, it might be “For those five governors who have had 40 or more fact-checks ruled on by PolitiFact, fact-checks on Scott Walker show him to have the lowest Truth Index of all five, with Chris Christie at Number 3." She could also say if you look at the Mostly False and False categories (and leave out the somewhat controversial Pants on Fire), Scott Walker is WAY ahead of the other five: while the other four average about 33% in these two categories, Walker is at 55%.  

But who’s going to say that.