Politi-Score: Jumping the Snark (Part 1)

Michele Bachmann will be my test baby for the first in a series of Politi-Psychotic pieces examining and analyzing the results of multiple fact-checkers on an individual politician’s statements.

Bachmann has been getting more reviews lately because of her presidential inclinations, and there are four fact-check sources to consider on her, and “consolidate” into one. A total of 50 Bachmann statements (all of which were made during 2011) are broken down as follows in parentheses: PolitiFact (15), FactCheck.org (15), the Washington Post Fact Checker (13), and the Minnesota Public Radio (MPR) PoliGraph (7).

After categorizing the statements into their levels of truthfulness, I realized I may have to revive the Politi-Score just retired in favor of PolitiFact’s Truth Index. It could be said that the Politi-Score has undergone a complete re-design and emerged as a comprehensive, “global” Truth Index, taking into account other fact-checkers. So, look for that tag soon.

I tried to keep the rules consistent to those used in my last post on the Fact-Checker: FactCheck.org was the tough one, because it does not grant rulings. So, there were basically four categories: True (which, like Glenn Kessler’s Fact Checker, are very rarely awarded); Inconclusive (I don’t think FactCheck.org would even publish those), which would rank a Half True; Misleading, which would get a Mostly False, and False, and this is where FactCheck.org uses the words “False” or “Inaccurate” (untrue, erroneous) somewhere in its review describing the statement at hand. If it does not use those words, it falls into the Misleading or Mostly False category. I did do one “Mostly True” however for FactCheck. This was a numbers claim which fell within the percentage “minimum” for Mostly True that’s been posted about before in a review of PolitiFact numbers claims. The MPR PoliGraph was handled similarly to the Washington Post Fact Checker-PolitiFact Truth Index in my previous post.

Of the 50 statements, one third, or 16 were duplicates (speaking of "selection bias"). That is, there were 10 statements which were also evaluated and ruled on by another fact-checker. There were two statements reviewed by all four fact-checkers (6 dupes). There were two statements reviewed by three of the fact-checkers (4 dupes). The other six were reviewed by two fact-checkers (6 dupes). Those I was most interested in examining were ones which could have (widely) divergent rulings. However, there were cases where the rulings had to be reconciled with the methods of the Fact-Checker and the context they said they examining, in other words, they may not be as divergent as thought.

Here is Bachmann’s Truth Index for each Fact-Checker, with the newly appointed “Politi-Score” at the end, which is the overall average:

I have to figure out a better way to present these graphs:  CLICK on graphic TO ENLARGE.

In my next post, I will take a look at some of the individual duplicated statements and the discrepancies between rulings between and among the fact-checkers.