Unless PolitiFact’s (PF) writers publish a flurry of rulings today (yea, right!), I now have some final figures for PolitiFact's Truth-o-Meter ratings through the end of 2011. As I’ve done before, I'll start with “everything including the kitchen sink” and then I’ll be honing down to just 2011 and more detailed analysis. I also plan to publish something on “best/worst/most/least” of the year 2011.
So here's the Truth Index in its "cool" and 3-D PolitiFact app format, in a way PolitiFact won't show it: of the 4,707 statements, 2,475 (52.6%) were by Republicans/ Conservatives/Tea Party/Libertarians; 1,863 (39.6%) were by Democrats/Liberals, and 369 were by independents, non-partisan or unknown political affiliation. These statements include all those made by individual politicians, as well as pundits, advocates, media, groups, websites, PACs, blogs and chain e-mails.
 |
| Click to enlarge |
There are some who don’t like PolitiFact’s signature “Pants on Fire” designation because of its subjectivity—deciding what makes a statement go from False to “ridiculously” False. So, just for sh*ts and giggles, here’s what it would look like if all the Pants on Fire statements were moved to False.
 |
| Click to enlarge: Pants on Fire rulings are included in False. |
The Republican’s overall Truth Index improves (from a negative 23 to a negative 18); the Democrats improve by fewer points (from just under a positive 1 [.7] to 3.4) but a greater percentage. Just a third of their statements are false, for Democrats it’s a little over one fifth or 22 percent.
Relating to this post, and to one of my blog’s main features (the Truth Index and Politi-Score), a FaceBook comment piqued my interest for a number of reasons. It was made by someone with an interesting shark profile icon by the name of Robert Hannah, kissing up to my conservative counterpart about his generic, ad nauseum comment concerning PolitiFact’s “selection bias!” in regard to Rick Santorum’s Truth-o-Meter review:
@Bryan exactly! I personally think he's a lunatic, but these report cards are meaningless. They're only going to rate statements that are controversial or divisive. So basically, out of the controversial things that Santorum has said that have attracted the attention of PF, most are untrue or half true.
Of course, since Hannah is profusely agreeing with Bryan, he has conveniently neglected responding up to this point. The Facebook format only displays these final two comments on the page (see graphic) which makes them readily viewable which I suspect is why he’s not besides the fact that Hannah is not disagreeing—he’s automatically “smart” and “right” in Bryan’s world. No need to copy/paste/parse and repeat as quickly as possible until oblivion, or make projecting complaints of “personal attacks.”
 |
| Click to enlarge: At least Hannah explains it. |
Yes, PolitiFact only rates statements that interest them, or what may be considered “controversial or divisive” but after so many statements, a definite pattern emerges. And when applied on a macro level, for example, rating 2,475 “controversial” statements by Republicans as compared to 1,863 “controversial” statements by Democrats, the patterns become even more unambiguous. At this level, the measure of “truthiness” has some meaning, whether or not you believe there’s selection bias or journalistic bias.
What Hannah is assuming is that if the uninteresting statements are rated and averaged in, everyone would be equal, or any differences would be statistically insignificant. He’s claiming that the sample PolitiFact takes (as well as the other fact-checkers) is insufficient and basically corrupted (as does Bryan White). The other issue is, in so far as PolitiFact has selected over 600 more statements by Republicans, is it because Republicans as a rule make more controversial and divisive statements? Or is it just “selection bias” as Bryan White would have you believe?
I would venture to bet that if you could take everything one politico said on a public basis, took an average and removed the rated statements, you’d get a much, much higher Truth Index. You might even get a higher average on the Republican side. But when you averaged the rated statements back in, there would be a similar pattern to what the Truth-o-Meter rulings have already established. In other words, truthfulness can’t be solely determined to the exclusion of controversial and divisive statements.
As I’ve discovered in reviewing statements rated by FactCheck.Org, the fact-checkers are more interested in statements that are more likely false. If they only rated True and Mostly True statements, they’d be “out of business” in certain respects, and most people would not find the fact-checkers very interesting.
Near the end of February, 2012, PolitiFact will be surpassing the milestone of 5,000 Truth-o-Meter rulings: I don’t know how 5,000 rulings could be called statistically meaningless.
No comments:
Post a Comment