One Way PolitiFact Contends with "Selection Bias"

Coming from that exciting (ha ha ha) world of accounting, the audit or financial internal control function served to police those who kept the books, from committing fraud or gross accounting malpractice.  There were certain procedures that had to put into place:  one was not to let one person control too much of the process, and to transfer employees between jobs frequently--even to compel them to go on vacation because often when the employee was away, such fraud or malpractice would be discovered.

 

This analogy is of course not about "fraud" in relation to PolitiFact, but about how a journalist who in the same way might write all fact-checks for one person (on one topic) might encourage the potential for the selection bias that is so often cited by its many critics.  And in the course of my recording over 6,500 PolitiFact rulings I noticed a pattern in assignment of the fact-checks in that no one writer dominated covering any one person or subject.  I have previously posted that writer Lou Jacobson appeared to be the "economics" guy, but economics has a wide umbrella covering many subject areas, such as jobs, taxes and the federal budget, even healthcare.  For example, a selection of the 13 consecutive rulings on Obama under the subject of taxes, out of 121 total rulings on Obama in 2012, shows this alternating pattern of ruling assignment quite clearly:

Click to enlarge:  Note column for writers. 

Even in the PolitiFact states, alternating topics and who was assigned the fact-check seemed to be the norm.  Here's an example from the  fact-checks by PolitiFact Wisconsin of its most frequently checked person Governor Scott Walker, on one of its most frequently checked topics, the state budget:

Click to enlarge

On occasion where there was a speech or television interview with a lot of statements to check, we might see a concentration of fact-checks by one writer, sometimes as many as five, but this was rare.  Eventually another writer would take the reins.  After the election in some of the states, it appeared that a single writer took over until the new year; then it was back to alternating.

Of course, I can't say that this resolves the problem of selection bias.  The type of bias that they're being accused of having--which involves a liberal partisanship--would still be evident regardless of how and to whom a fact-check is assigned, if it could be proved.  But I would think it would be somewhat ameloriated through multiple journalists covering multiple topics, because not everyone has the same bias going in the same direction.