Truth Index+ by PolitiFact National and States

The state by state Truth Index+ has held pretty consistently since I first started crunching the data last year (even using the PolitiFact calculation).  Now I can present it (with a great deal more accuracy) for all rulings since 2007 through the eve of the 2012 general election on November 6.

 

Click to enlarge

That trend holds overall with one exception: that the Truth Index+ for Democrats exceeds that of Republicans.  Overall the variance is 15.9%.  To express it appropriately, the statements selected for Democrats by PolitiFact National as well as all state PolitiFacts have been on average rated about 16% higher in terms of more True/Mostly True/Half True(s) and fewer Mostly Falses/False/Pants on Fire(s) on PolitiFact's Truth-o-Meter.  Also, if you designate the Pants on Fire as equivalent to False instead of rating it lower (that is, remove the effect on the Truth Index score of Pants on Fire), this variance decreases to 13.9%.

 

At the state level, the Truth Index+ variance between Republicans and Democrats ranges from a nearly dead even 1.4% at PolitiFact Virginia, to 27.7% at PolitiFact Tennessee, although there it may be an anomaly due to the fact it had the lowest number of rulings measured (73 total) as the newest PolitiFact state franchise.  And the exception is PolitiFact Wisconsin at a "negative" 7.2%, that is, their Truth Index+ is the only one favoring the Republicans.

 

The anti-PolitiFact propaganda website PolitiFact Bias (PFB) recently posted its 2012 Pants on Fire factor (how much "more" it claims Pants on Fire are assigned to Republicans) by PolitiFact state.   And it has discovered via the results it shows, as I've reported before, some of the states actually assign more Pants on Fire to Democrats.

 

In its original research paper, we are told that the factor is adjusted for chain e-mail and "intra party" claims.   Because of the primaries this year, there were a lot of such "intra party claims", and due to that, although I go in the general direction of PFB's calculations of the factor, I was not able to match it.

 

The state where I was most off PFB's calculations was Georgia, which had a total of 18 False and 8 Pants on Fire claims for 2012 through the election, of which only 3 were for the Democrats.  My "raw" factor was 1.11 against the Democrats, while theirs was 6.5 against the Democrats.  With so few Democrat rulings, you'd think it would be easy to match PFB's factor.  But even after adjusting for those rulings which appear to correspond to PFB's criteria, I did get to a factor of 5.0, but could not determine how it got to the 6.5.  (I re-verified my total counts for PF Georgia, and they were right on.)

 

In view of the above, I will only publish the "raw" Pants on Fire factor here (shown below).  That is, it's not adjusted for anything.   And instead of looking at just the first 10 months of 2012 (as PFB did), here it is for PolitiFact National and its state franchises since it started back in 2007: that's 1,893 False and Pants on Fire rulings (29.6%, or 561, of the False rulings were "elevated" to Pants on Fire).

Click to enlarge:  There's very little correlation between this factor and the Truth Index.

Comparing to the Truth Index chart above, we can see that there are five PolitiFact states (out of 12 including National) who assign more Pants on Fire to Democrats, and yet four of those five (Georgia, New Hampshire, Tennessee and Virginia) have a Democrat-favorable Truth Index overall.  Only PolitiFact Wisconsin seems to go the way one would expect.  So while the Democrats may make more Pants on Fire statements, they also have more True and Mostly True statements to offset them.  At least in those four states.

 

As to why this is so, a few studies have come to various conclusions.  The difference at PolitiFact Wisconsin, and the early and later (2007-08 to 2009 forward) PolitiFact averages, may provide a clue.  In the case of Wisconsin, it appears to be the recall elections of 2011, and in the case of "post-early" PolitiFact,  it appears to be....Obama and Obamacare.  In both cases the opposing party--the party out of power--fought a loud and contentious battle against the legislation of the other party, and in the course of doing so damaged "the truth."    The "loud and contentious" included those statements that "piqued the attention" of PolitiFact writers.

 

As far as "selection bias" I am trying to characterize the Truth Index in terms that include the caveat of such bias.  But that is only how far as I will go, as there really is no evidence that it is the only factor that is causing the Truth Index to favor the Democrats.  Next, the major PolitiFact writers get the Truth Index treatment--and maybe with the Pants on Fire number too.