Truth Index+: Adjusting for Pants on Fire

The Pants on Fire factor as developed by anti-PolitiFact propaganda website PolitiFact Bias (PFB) leaves much to be desired--specifically, it does not take the other four rulings (True through Mostly False) into account.  Their reason is the more "subjective" nature of the Pants on Fire ruling as defined by PolitiFact itself (not just inaccurate but "ridiculous").  Basically it measures how much more (or less) Pants on Fire is assigned to Republicans when a ruling is false.  But when the Pants on Fire are taken out completely (and are calculated as False), the rulings still favor the Democrats on average about 17% (for all 2012 rulings through election day) using PFB's own baseline methodology (adjusting for intra-party, chain e-mails, pundits, PACs and party boosters).  Regardless, the Pants on Fire effect is very minimal, as recently pointed out by Brash Equilibrium's Malark-o-Meter and in an earlier review I did in 2011.

A better method might measure the overall percentage change in the (newly formulated) Truth Index+ when the Pants on Fire are removed and categorized with the same score as False.  In the example data above, the Democrats' Truth Index+ is improved 1.4% by eliminating Pants on Fire, while the Republicans Truth Index+ is improved 3.5%.  This percentage could be called the Pants on Fire Bias factor.

One could then either take the remaining Republican to Democrat difference in the Pants on Fire adjusted Truth Index, which in this case is 16.8%, as what would be the undefined or non-subjective "bias" variance.  One could also measure the difference between the 1.4% and the 3.5% which would be about 2.5.  This is a Pants on Fire bias factor which incorporates the other rulings.

In other words, in terms of ALL rulings True through Pants on Fire, the Republican Truth Index is effected 2.5 times as much by Pants on Fire than is the Truth Index of the Democrats.  All because Republicans are assigned Pants on Fire about 74% more than Democrats when a ruling is false (to use the PFB factor).

It's ironic that I came out to 16.8% since Brash Equilibrium's statistical testing of a sample of PolitiFact rulings hypothesized there may be as much as 17% variance in the rulings due to bias.  But that is a qualified "as much as" so it doesn't really tell us anything.  More to follow as I apply this to all rulings through election day, 2012.