Truth Index+: The Top Ten PolitiFact Writers

There are ten PolitiFact (PF) writers who have produced just about half of PolitiFact's rulings, from its beginnings in 2007 up until I ended my compilation on Election Eve, 2012.  Another roughly 160 writers composed the rest of them, many serving multiple functions of writer, researcher and editor.  There were 47 writers who wrote just one to two rulings each.

 

The highest producer by far is Lou Jacobson, whose 676 fact-checks (some done in association with other writers at PolitiFact) make up a bit over 10% of PolitiFact's total output.  Number 2 is Angie Drobnic Holan, now the Editor at PolitiFact Florida, at 571.

 

Below is the Truth Index+  for the Top Ten, with scores for Republican versus Democrat.  The best way to look at these averages in terms of "bias" or "lean" would be the percent variance in the Truth Index+ between party for each writer.  Lou Jacobson, Angie Drobnic Holan, Robert Farley and PF Georgia's Willoughby Mariano have about a 22% variance leaning toward the Democrats, only to be exceeded by Number One C. Eugene Emery of PolitiFact Rhode Island, who has a whopping 38% Democrat lean.   Tom Kertscher at PolitiFact Wisconsin swings 22% in the opposite direction; the remainder go from 3% in favor of the Republicans to around 10% in favor of the Democrats.

 

Click to enlarge:  NA in parentheses = PF National; then number of rulings written.

So, what can we learn from the Truth Index+ for each writer?  I decided to take a closer look at the top four writers and an "outlier" in terms of some of the items I had keyed in:  party, titles, people, subjects.

 

Lou Jacobson, for example, was No. 5 in "lean" variance as noted above, although other than Emery he was in a virtual tie with four others.  But he had a few features in his 676 rulings that stood out:  he is definitely "the economics guy" on the PolitiFact team.   The subject of about 40% of his rulings was a fiscal one: economy, federal budget, stimulus, taxes etc..  This was a far larger percentage than any other writer.  Healthcare was the subject of 25% of his rulings.    In terms of "occupation" his being on PolitiFact National's team meant that about 30% of his rulings were on the president and vice president and the presidential/vice presidential candidates (in 2008 and 2012).

 

A check on names revealed a notable person who seems one of Jacobson's "fact-check favorites"--Michele Bachmann.  He did almost a third (17) of her 54 rulings, and gave her 5 of her 12 Pants on Fire awards, as well as five Falses (in other words, when a claim was found False for Bachmann, Jacobson determined it was Pants on Fire half of the time.)  In comparison, Angie Drobnic Holan only gave Bachmann one Pants on Fire out of five rulings found False.

 

Angie Drobnic Holan scored even higher than Jacobson as far as "occupation"--42 percent of her rulings covered the president, vice president, and presidential/vice presidential candidates.  While I posted previously that she (and writer Catherine Richert) were the "healthcare" writers in 2009, about 25% of her rulings in total covered that subject, with another 25% on fiscal issues.  Of note, she did roughly a quarter of all chain e-mails.  She wrote 27% of all chain e-mails awarded a Pants on Fire.

Robert Farley was with PolitiFact from January of 2008 until July of 2011, and now writes for Factcheck.Org.  He produced 432 rulings during that time.  He did about the same percentage (31%) of presidential contenders as Jacobson, and about 25% of his fact-checks on fiscal issues, like Holan.  There wasn't much of anything that stood out ruling-wise on Farley.

W. Gardner Selby of PolitiFact Texas, now its editor,  did 338 rulings since PF Texas came on the scene early in 2010.  He also did about 25% of his rulings on fiscal issues, and as he's from Texas, he's covered governor Rick Perry on about one third of his rulings.

While the National writers all have almost the same percentage between Republicans and Democrats on the Truth Index (about 22%), Selby's variance is just under 10%.   And there are some good reasons why.  Selby's rulings relating to the presidential election, even with Ron Paul and the brief candidacy of Rick Perry, were less than 10 percent of the rulings.  Fifty-five percent related to local issues which are less ideological.  Only 6 percent related to healthcare, one of PolitiFact's most controversial and often-graded topics due to Obamacare.

Another factor that was found to occur across the board for these four writers who've collectively done about 30% of PolitiFact rulings is their percent of coverage between Republicans and Democrats.  Jacobson, Farley and Selby had about 56% Republican statements "selected" while Holan was at almost exactly half.  Jacobson and Farley had about 40% Democrat statements, Holan 46% and Selby 32.5%.  This seems to say, particularly for Republicans, that if there's "selection bias" going on, it may be there in terms of "let's try to keep Republican statements at about half of what we choose to look at."

Lastly, I'd like to take a look at the GOP-favoring Truth Index+ outlier, Tom Kertscher of PolitiFact Wisconsin.  The two PolitiFact Wisconsin writers Tom Kertscher and Dave Umhoefer had the lowest overall Truth Index of the top ten group.  Tom Kertscher's was the second lowest at 43.3 (Umhoefer third).  Keeping with the "half Republican" selection criteria above, Kertscher had an almost equal division of Democrats versus Republicans in terms of number of rulings (98 Democrat rulings, 97 Republican).  He did only one ruling on Obama (over two years ago) and one on Wisconsin congressman Paul Ryan as the new vice presidential candidate.   Although he covered fiscal subjects at a similar percentage to the others (28%), he hardly covered the more controversial healthcare issues (12% of his rulings) the way the other writers did.

In terms of the "major," more well-known candidates in Wisconsin, Kertscher's rulings tended to cover Republicans more often.  We might expect that with Scott Walker as the winning gubernatorial candidate: Kertscher covered him the most, two to one over his competitor Milwaukee Mayor Tom Barrett.  His Truth Index favored Walker by almost 35%:  Walker was averaging a respectable 54.1, Barrett at a precise "Mostly False" 40.0 (although he did only 8 rulings on Barrett).  Interestingly, the "Pants on Fire" factor was 1.67 in favor of Barrett, despite his low Truth Index.

Looking at another race between senatorial candidates Tammy Baldwin and Tommy Thompson, Kertscher again covered the Republican Thompson about two to one, with 6 rulings on Baldwin and 14 on Thompson.    The Truth Index+ for both reflected Kertscher's general negative lean: Baldwin averaged 46.7 for her 6 rulings, while Thompson was in Barrett's camp at "Mostly False" 40.0.  In the case of the Pants on Fire Factor, Kertscher spared Baldwin, and only rated Thompson that way once.

Overall, Kertscher gave out twice as many Pants on Fires to Democrats during his tenure at PolitiFact Wisconsin.  He also gave fewer "Trues" to the Democrats (the most Trues were  to Scott Walker).  Earlier this year, his rulings were part of my post reviewing some recall election-related fact-checks suggesting that there may have been a right-wing bias at PolitiFact Wisconsin.  

Kertscher covered fiscal issues at about the same average as the top PolitiFact National writers, at 30%.  He also has the distinction of having done zero chain e-mail fact-checks. If my conservative counterpart wanted to make a case about selection bias, Kertscher (and Umhoefer) might be a good place to start (for comparative purposes), or about how more national, ideological or controversial  issues appear to be behind the "liberal lean" of PolitiFact, since they weren't nearly as prominent in Kertscher's rulings.