Truth Index+: Are Election Winners More "Truthful"?

Since PolitiFact has been through three national election cycles and a few minor ones, I've always been curious as to whether the fact-checking matters in regard to who gets elected. Or to put it another way, are those who win more "truthier" according to the Truth Index.... or less? So as an experiment, I went through the PolitiFact rulings and keyed in whether the candidate/official won or lost in a pending election from the time he or she made the statement.

But first I had to do a lot of filtering of my list of 6,535 rulings (all rulings through 11/5/12).  All the non-official people (like pundits and advocates) and groups (like websites, bloggers, chain e-mails, PACs etc.) had to go, of course. I also didn't include (or partially included) the rulings of those who retired or lost in the primaries (good example: Dennis Kucinich). There were some who won and lost, like John McCain: I rated his statements prior to his 2008 presidential election loss as a "loser" and those since as a "winner" since he won re-election as Arizona senator in 2010. Local and state elections are also excluded except for governors races. And just for the heck of it, I also keyed in those who were in the Tea Party caucus. When I got done, I was down to 2,937 rulings.

Click to enlarge:  Winners appear to make fewer False/PoF statements regardless of affiliation.

As of November 5, 2012, for all 6,535 PolitiFact rulings, the Truth Index+ is 54.2. For the 2,937 candidates making campaign statements, it was about 5 percent higher at 57. However, the "all" includes everything along with the kitchen sink. A better comparison might be to the Truth Index for those were in office at the time of the statement, which is similar to those facing elections, which was a tiny bit higher 57.7. Compared to the overall, however, what this says is that individuals and those in official positions tend to be more truthful on their own than groups or those who advocate for them. I've already shown this to be the case when I've posted calculations of the Truth Index separately for office-holders versus the various categories of advocacy groups, party boosters, media pundits, etc.

The "bias" factor would not be as relevant here because I am not looking at this in terms of Democrat versus Republican. But it's evident from these statistics that the "winners" are found more "truthful" by the PolitiFact fact-checkers than are the "losers" by about 8 percent in terms of the Truth Index+. It's about 6 percent if you re-adjust the Pants on Fire category to False.

Comparing Republicans and Democrats gets a wee bit trickier. Democrats have always held a higher Truth Index in general, and even though some say it's due to selection bias, something did emerge from the analysis of interest concerning the Republicans that can't be totally attributed to bias. While the Democrats had a pretty substantive variance between winners and losers (63.6 versus 55.8, or 14 percent), the Republican's variance was insignificantly small--53.5 versus 52.8, or about 1.5 percent. What this says is that the Republicans have about the same level of "truthiness" based on statements evaluated whether they win or lose. What this means, I don't know.

As far as the statistics for the Tea Party Caucus members, I don't have enough rulings for those Tea Party members who have lost (and yes, I included statements by Allen West!). So far they have had mostly winners and their winners statements still far outnumber their losers statements. For the record, the Truth Index for Tea Party winners' statements separated out of the Republican index is a dismal 40.4 (because over 50% of their statements were rated False or Pants on Fire which might have something to do with Michele Bachmann's inclusion in this group). Taking Tea Party statements out of the Republican average for winners versus losers, however, changes the variance from 1.5 percent to 5.2 percent (55.8 to 53). But even if the Tea Party breaks up, I would venture that some of them would still be around as Republicans, as would the almost equivalent Truth Index between Republican winners versus losers.

So we may be able to conclude from this that yes, there's a better outcome election-wise for those who are found more "factually correct", Republican or Democrat. The only anomaly is the Tea Party group, who have had tremendous success winning elections (even in 2012) but have very low Truth Index scores relative to both Republicans and Democrats. Right now their Winning index is lower than their losing index, but for their losing index there's only 24 statements, so not much can be concluded for winning versus losing.

As usual, a note of caution about the Truth Index as well, as quoted from a Truth Index-related article posted by poll guru Nate Silver's column FiveThirtyEight.com:

They aren’t perfect indicators of the honesty of each candidate, and conclusions like “Candidate X lies the most” or “Candidate Y is the most truthful” should probably not be drawn from the data.

The same goes for the parties during campaign season.