There's certainly a lot of people out there scoring PolitiFact rulings... I'm finding
more and more of them every day. Presented
and scored in different ways, in studies, articles and blogs, I think it may be
time to add them as a special section to my PolitiFact Detractors page, though "detractors"
are not quite what they are.
While I hope
I'm one of the better ones, I do need to make the case. I have probably looked at these rulings in the
largest variety of ways, and have done so on an ongoing basis for quite a
while (since 2010, when I started this blog). I've looked at the score in terms
of writer, topic, local versus national, by individual compared with other fact-checkers, etc. I'm looking for
patterns. For example, if Republicans
lie more, do they lie about every subject?
(No, there are some subjects for which they are more truthful than Democrats) Are lies more ideologically based? (Yes) Do some PolitiFact writers appear to favor one
party over the other based on the Truth Index? (Yes) Most of the other
PolitiFact quants have looked at them in terms of just the presidential
contenders or Democrat versus Republican overall. Some qualify their rulings as far as
sampling/selection bias while others take the rulings at face value.
Straight
Politi-Quant websites. The website Who's More Full of Sh*t (authored by J. Kalgreen) graphically rates the same....with an ongoing feed from PolitiFact. It
uses a score heavily weighted to the negative rulings. It also gives an overall Democrat versus
Republican rating. While it's a pure
rating website, its website set up is quite sophisticated. As noted by Benjamin "Brash Equilibrium" Chabot-Hanowell, "I seriously
wish I could code CSS and PHP as well as J. Kalgreen [WMFoS author] can."
 |
| While a great site, Who's More Full of Sh*t might not be linked by PolitiFact because its name might offend some. |
Instead of
saying who's more "truthier" or "honest" it goes in the
opposite direction--scoring zero for True down to 200% for Pants on Fire.
So if my conservative counterpart decided to try to troll-comment the site
(which he can't because no comments are enabled) he'd have to complain that
PolitiFact's selection bias is the cause for Democrats being less full of sh*t,
or for Republicans to be more full of sh*t.
Because, you see, both are still full of sh*t.
Malark-o-Meter
is more of the nerdy Nate Silver variety of
meta/probability analysis and in that respect is like no other
PolitiFact quantifier. A newer website
based on (I'm guessing) Joe Biden's outburst at the vice presidential debate, author
Benjamin "Brash Equilibrium" Chabot-Hanowell is interested in
mathematically determining a possible "bias" factor as well as the
probability of how much each claimant is prone to lie. He includes the Washington Post Factchecker rulings in his analysis as well.
"Serious"
Studies. I've posted about and often referred
often to Eric Ostermeier's 2010 study of 511 PolitiFact rulings where he
concluded with the infamous selection bias statement.
The Center
for Media and Public Affairs did a study of both PolitiFact and Washington Post
Factchecker rulings but did not really reach any other conclusion ("[took] no position on their accuracy") as to why
Republicans score as less truthful than Democrats.
And of
course, the website PolitiFact Bias did a research paper on PolitiFacts False/Pants on Fire
rulings, coming up with a percentage factor that I often use in my blog. There are others who have looked at the
Pants on Fire, some in a humorous way, or a lot like PolitiFact Bias.
Website Articles. One of the very first to quantify
PolitiFact's Truth-o-Meter rulings was the pseudonymous "Cinematical"
at the Daily Kos, back in 2009....and arrive at the conclusion as so many that would
follow, that conservatives were less "honest" than liberals. There's also been several scorings at the Daily Beast, as well as a nice quant review by the New York Times'
FiveThirtyEight.com (Nate Silver's blog).
Blogs. Anyone math-oriented knew the moment they saw
the Truth-o-Meter that it begged to be measured.
I imagine there's a lot more than just those mentioned here. So (besides myself), here's three very good ones:
In 2011, I came across the blog Quibbling Potatoes authored by Steve, who had some great ideas for categorizations
of the rulings that I now use for my data.
He also had a name for his score which he called...the Truth Index. While I don't think PolitiFact took the name
from Steve's site, one can certainly say Steve had a bit of clairvoyance. It's obvious he put a lot of work into his
charts and graphs and it would be nice to see him do it again.
Pandering Politicians is a blog (turned website like PolitiFact Bias) by someone who calls himself
"PanderBear" and the Truth Index here is called the "Truth
Quotient." He has a lot of charts,
especially time trendlines, and begs people not to even read chain e-mails
because of their dismal "TQ."
And of course, in one of the comment sections we see the ubiquitous question
from my conservative counterpart asking what he does about selection bias. No answer from PanderBear. At least as far as I could see.
Allen
Allermann in a namesake blog entitled "Random Human Neural Firings" did
a quantifying review about the same time as Steve called "Who do You Trust?" He scored PolitiFact rulings in the opposite
direction that many tend to do--he gave higher weight to the True and Mostly
True instead of False and Pants on Fire.
He uses a "sports" analogy for what Brash Equilibrium terms
the report cards of those rated by PolitiFact; he calls them "at
bats."
So there we
have, including my own, about a dozen attempts to quantify PolitiFact rulings. Perhaps two qualifiers can be added: (1) None
of these websites/studies/blogs are affiliated with or approved by PolitiFact (as
far as I know) and (2) all numbers should be interpreted with caution. And if you come across any websites/articles/blogs that quantify PolitiFact's Truth-o-Meter rulings, please let me know in the comment section.
