Truth Index+: The Top Ten PolitiFact Writers

There are ten PolitiFact (PF) writers who have produced just about half of PolitiFact's rulings, from its beginnings in 2007 up until I ended my compilation on Election Eve, 2012.  Another roughly 160 writers composed the rest of them, many serving multiple functions of writer, researcher and editor.  There were 47 writers who wrote just one to two rulings each.

 

The highest producer by far is Lou Jacobson, whose 676 fact-checks (some done in association with other writers at PolitiFact) make up a bit over 10% of PolitiFact's total output.  Number 2 is Angie Drobnic Holan, now the Editor at PolitiFact Florida, at 571.

 

Below is the Truth Index+  for the Top Ten, with scores for Republican versus Democrat.  The best way to look at these averages in terms of "bias" or "lean" would be the percent variance in the Truth Index+ between party for each writer.  Lou Jacobson, Angie Drobnic Holan, Robert Farley and PF Georgia's Willoughby Mariano have about a 22% variance leaning toward the Democrats, only to be exceeded by Number One C. Eugene Emery of PolitiFact Rhode Island, who has a whopping 38% Democrat lean.   Tom Kertscher at PolitiFact Wisconsin swings 22% in the opposite direction; the remainder go from 3% in favor of the Republicans to around 10% in favor of the Democrats.

 

Click to enlarge:  NA in parentheses = PF National; then number of rulings written.

So, what can we learn from the Truth Index+ for each writer?  I decided to take a closer look at the top four writers and an "outlier" in terms of some of the items I had keyed in:  party, titles, people, subjects.

 

Lou Jacobson, for example, was No. 5 in "lean" variance as noted above, although other than Emery he was in a virtual tie with four others.  But he had a few features in his 676 rulings that stood out:  he is definitely "the economics guy" on the PolitiFact team.   The subject of about 40% of his rulings was a fiscal one: economy, federal budget, stimulus, taxes etc..  This was a far larger percentage than any other writer.  Healthcare was the subject of 25% of his rulings.    In terms of "occupation" his being on PolitiFact National's team meant that about 30% of his rulings were on the president and vice president and the presidential/vice presidential candidates (in 2008 and 2012).

 

A check on names revealed a notable person who seems one of Jacobson's "fact-check favorites"--Michele Bachmann.  He did almost a third (17) of her 54 rulings, and gave her 5 of her 12 Pants on Fire awards, as well as five Falses (in other words, when a claim was found False for Bachmann, Jacobson determined it was Pants on Fire half of the time.)  In comparison, Angie Drobnic Holan only gave Bachmann one Pants on Fire out of five rulings found False.

Pants on Fire Factor: Obama-Fact

The addition of the state PolitiFacts began with the appearance of PolitiFact Florida in 2009, but really didn't really get in gear with publishing a substantive number of rulings until 2010.  In fact, in 2009, PolitiFact "Florida" published a grand total of 9 rulings, bringing in lead PolitiFact Florida writer Aaron Sharockman around October.  Then things took off in 2010 as Texas, Georgia, Rhode Island were added in the first half and then Ohio, Oregon, Wisconsin and Virginia in the second half.

 

It would seem as if the states came along to fill in the gap of the Congressional Quarterly, whose writers worked with PolitiFact doing rulings from 2007 through 2008.  However, as I described in my original assessment of this group and what its "affects" may have been , the types of statements selected at that time were of a different nature:  in the case of Congressional Quarterly, they related to its area of expertise, congressional positions and votes (although they did do other types).  In the case of the PolitiFact states, however, many rulings had to do with matters of a local nature, for example, one of the first PolitiFact Florida rulings on  a statement by then-Governor Charlie Crist who was bragging up the reduced rates of a negotiated, un-subsidized Florida healthcare program called Cover Florida Healthcare (found False).

 

The "Pants on Fire factor" (PoF) as I call it, a measure of what is claimed to be PolitiFact's bias for rating a False ruling as Pants on Fire, is shown below as aggregated from 2007 to September of 2012.    It is a comparison of Democrats to Republicans, with a factor of 1.0 translating to mean that PolitiFact assigns Pants on Fire to Democrats at the same (or equal) rate it assigns Republicans.  Anything below the 1.0 means Republicans are assigned Pants on Fire less than Democrats, anything above the 1.0 means Republicans are assigned Pants on Fire more than the Democrats (as a percentage of False + Pants on Fire rulings).

 

 

The red line shows the growth of the factor from favoring the Republicans to favoring the Democrats (or growing ever more negative for Republicans) over PolitiFact National's six years of Truth-o-Meter rulings.  The chart begins in quarterly increments until the end of 2008 to show how the Congressional Quarterly scored for that period as well, displayed in the green line. Thereafter the increments are annual.

 

At the end of 2008, PolitiFact National (including the Congressional Quarterly) had a factor below 1.0, although as shown by the green line, the Congressional Quarterly (CQ) alone was assigning Pants on Fire to Republicans 42% more than to Democrats.  With CQ behind them, in 2009 the PoF factor made its biggest jump ever at PolitiFact National, from .90 to 1.60.    For the year 2009 alone, PolitiFact National's Pants on Fire factor was a whopping 5.06.

 

Accumulated through the end of September of 2012, it now stands at 1.94--in other words, in the course of its history, PolitiFact National assigned Pants on Fire almost twice as often for Republicans as it did for Democrats.

 

Combined with the state PolitiFacts, however, the chart takes a different slope, as shown by the purple line.  It starts at the same high point as in 2009 because of there being only 9 PolitiFact Florida rulings, but as they accumulated, the aggregate Pants on Fire factor for PolitiFact as a whole dropped, from the 1.60 in 2009 down to its current 1.33.  The states themselves, if they were separated from PolitiFact National, would now average 1.21.

 

So it is indeed true that once PolitiFact "dissolved its partnership" with Congressional Quarterly, it began to appear as though it was increasingly assigning Falses as Pants on Fire to Republicans.  But working with CQ may not be a cause even with such a correlation, given that its own writers' assignment of Pants on Fire to Republicans were far more (58% more) than that of what PolitiFact National was doing at the end of 2008.  Were there other possible reasons?  What happened in 2009?

Politi-Score: First Half Preview--PolitiFact Writers Revisited

The first half of 2011 is almost over, so I have started compiling PolitiFact rulings from where I left off at the end of March. As I start with PolitiFact National and there’s just a few days left in June, I thought it would be worth it to take another look at the PolitiFact National writers again in terms of the Politi-Score. At that time, it appeared that PolitiFact writer Angie Drobnic Holan favored the Democrats more than the two other writers Lou Jacobson and Robert Farley; however, now that more rulings have been accumulated, that theory may have been a bit premature.

Holan’s Politi-Score dropped from 63 to 53 for Democrats and went up a bit for Republicans; both Farley and Jacobson dropped in “truthiness” on both sides. Here is the chart for the first quarter, and below that (outlined in red) is the chart for the first half (first and second quarters combined):

First quarter 2011 (click to enlarge).

First Half 2011 (click to enlarge).

Conclusions? Well, if we are to believe that PolitiFact is liberally biased, it would seem they selected more Republican statements in the second quarter that were false. They did select more Republican statements, aside from whether they “knew” the statements would be ruled False or Pants on Fire. For the record, there were 116 rulings total published by PolitiFact National for the second quarter of 2011 (as of Monday, June 27). Of these, 45 were Democratic/Liberal statements and 67 were Republican/Conservative (the rest were non-partisan). The Democrats had only two Pants on Fire rulings, and 10 False; the Republicans had 11 Pants on Fire rulings, and 19 False. That’s a ratio of 5:1 on Pants on Fire and 2:1 on False comparing Republicans to Democrats.

Yesterday, a Facebook commenter named Vic Pilkington addressed me on something similar:

… what is your tally when comparing right and left "pants-on-fire" and "false" starements? I contend the right have a 4:1 edge over the left on those statements considered as having no basis in reality.

Well, I guess it’s even worse than Vic thought. For the second quarter of 2011, at PolitiFact National, there IS some basis for that reality. On the other hand, we have about a quarter of all the rulings represented here, and the rest (the PolitiFact partners) need to be reviewed.

Politi-Score: Angie's List Left

Last January I did a post in which I looked at Bryan White's Grading PolitiFact series of criticisms which he believes establishes a "preponderance of anecdotal evidence" of PolitiFact's liberal bias.  The title of my particular post was "Does Lou Lou do More Doo Doo" because, as I noted in the piece, Bryan has never given a "grade" to PolitiFact's writer Louis Jacobson any higher than an F.  Indeed, it appears he looks at more PolitiFact rulings by Jacobson than anyone else, although my count at the time revealed that Louis Jacobson was tied overall with Robert Farley at 23 rulings each, while Angie Drobnic-Holan came in second with 18 rulings reviewed.

Starting this year in my "counts" I decided to include the writer in order to calculate a Politi-Score by PolitiFact writer.  The quarterly results offered a slight surprise:  it may be that Bryan is not looking at the right person.  Lou Lou doesn't do the do do as much as Angie does (if there's to be a suspicion of liberal bias), according to the Politi-Score:

Click to enlarge.

This is roughly 100 rulings: Farley and Holan did one quarter each, and Jacobson did the other half.  The Politi-Scores tell a story, however, with Angie Drobnic-Holan giving Democrats a very high (the highest of all three) Politi-Score of 63 compared to her two PolitiFact National co-workers.

At this point, all this tells me is I need to continue compiling results for the writers to get a larger sampling of rulings, which would give more validation to this result.  If I have the time I may go back and compile them for 2010.  Stay tuned.