As part of an analysis of PolitiFact’s (PF) “bias” I made a list of possible items to consider: one was, is there one PolitiFact writer/researcher who may be more biased or prone to error favorable to one side than the others? I thought Bryan White’s Grading PolitiFact series might provide some insight into that. I also wondered, now that I had a population numeric for PolitiFact rulings (2,788), did Bryan attain a number of grades that might be considered a sufficient sample. There was also a certain curiosity as to whether any other characteristics (who made the statements, etc.) of his sample revealed any other trends of interest.
So I went back to his blog and started adding up the rulings. I found that he “officially” started his Grading PolitiFact series in 2009 coinciding with the beginning of Barack Obama’s presidential term. So I counted two years forward from that, to January 20, 2011, getting a total of 97 rulings. According to statistical sampling techniques, he would need a sample of at least 142 rulings with the same finding of “bias” to achieve a 95% confidence level for that finding to exist in the rest of the population. Bryan also “graded” PolitiFact in 2008, so there are additional rulings on top of his 97. But I would make an educated guess that he doesn’t have 45 additional rulings to get to the 142 benchmark. Additionally we get brought down by an important (and perhaps most crucial) requirement of the sampling: it must be unbiased, and we know it is already tainted (because Bryan openly admits to his conservative bias).
Bryan predominantly selected Republican rulings for analysis (60%) with the highly-inflammatory Michele Bachmann and similarly noisy, opinionated pundits like Glenn Beck and Rush Limbaugh topping his list. His choice of Democrats follows a similar pattern of PolitiFact’s choices, with Barack Obama getting roughly one third of the Democrat ruling analyses.










