Politi-Score: Dumb Politics in Detail

I wasn’t quite satisfied with the analysis of my previous post on Smart Politics’ Eric Ostermeier’s critique on PolitiFact rulings. I decided to do something a little more in depth and graphical, and to look into my contention that he only used PolitiFact National’s ruling data. Some of this post repeats some of the findings, but puts them in a more understandable format. And, in the process, I found out I needed to make some adjustments to the database I had collected because of an interesting finding.

Of the 511 statements put through the Truth-O-Meter test from January 1, 2010 through January 31, 2011, PolitiFact devoted 74 percent of its attention to current and former political officeholders and elected officials (379 statements), 17 percent to ideological organizations and individuals not holding political office (85 statements), and 9 percent to other groups and individuals without a partisan or ideological agenda (28 statements). Another 20 statements came from chain e-mails, public opinion polls, bumper stickers, or "bloggers" generally (4 percent).

Source of PolitiFact Ruling (PolitiFact National)	Ostermeier Count	Ostermeier Percentage	Politi- Psychotics Count	Politi- Psychotics Percentage
Current and former political officeholders and elected politicians	379	74.2	1,330	74.6
Ideological organizations and individuals not holding political office	85	16.6	316	17.7
Other groups and individuals without a partisan or ideological agenda	28	5.5	34	1.9
Chain e-mails, public opinion polls, bumper stickers or “bloggers”	20	3.9	104	5.8

His numbers add to 512 (one more than 511), and the chain e-mails, etc., overlap with groups and individuals without a partisan or ideological agenda. In other words, when he said “9 percent to other groups and individuals” it includes the 20 statements from chain e-mails, etc., since his quoted 28 does not compute to 9 percent of 511. I’ve divided that 9 percent to show he had about 5 percent for those without an ideological agenda, and about 4 percent for chain e-mails, etc.

What is complicating is that those chain e-mails and public opinion polls can also be without a partisan or ideological agenda; so I assume his division ignores that. In my database of rulings, of the 104 chain e-mails, etc., 62 of them had an agenda which I could only determine to be unknown, that is, “without a partisan or ideological agenda.” Including them in Ostermeier’s category of “Other groups” would have brought my average more in line with his at 5.4%, and the “Chain e-mails, etc.” category would now be 2.6 percent, considerably lower than Ostermeier’s 3.9%. But it would not be ALL chain e-mails. Ostermeier should have made that clear. 

For those current or former political officeholders, PolitiFact has generally devoted an equal amount of time analyzing Republicans (191 statements, 50.4 percent) as they have Democrats (179 stories, 47.2 percent), with a handful of stories tracking statements by independents (9 stories, 2.4 percent).

This was one measure where we were very close, about 5 percent less for Republicans and 5 percent more for Democrats (from his calculations to mine). Overall, however, the Democrats had more rulings, compared to what was found by Ostermeier in his 13-month "2010" sample.

Party	Ostermeier Numbers	Ostermeier Percentages	Politi-Psychotics Numbers	Politi-Psychotics Percentages
Democrats	179	47.2	661	49.7
Republicans	191	50.4	638	48.0
Other	9	2.4	31	2.3
Totals	379	.	1,330	.

The False and Pants on Fire categories are where we have a fairly considerable divergence, although the Democrats still get almost half as many False and Pants on Fire rulings as Republicans. The Democrats and Republicans “traded” 12 percentage points compared to Ostermeier's numbers, according to my calculations.  The results follow Ostermeier's quote:

Republican statements were graded in the dreaded "false" and "pants on fire" categories 39 percent of the time, compared to just 12 percent for statements made by Democrats.

That means a supermajority of falsehoods documented by PolitiFact over the last year - 76 percent - were attributed to Republicans, with just 22 percent of such statements coming from Democrats.

Category	Democrat Count	Dem %age	Republican Count	Repub %age	Other Count	%age
Ostermeier Totals of False+ POF	.	12%	.	39%	.	.
(My) False	90	14%	152	23%	5	16%
(My) Pants on Fire	16	3%	50	8%	1	3%
Total	106	17%	202	31%	6	19%
Ostermeier % of All False+POF Rulings	.	22%	.	76%	.	2.2%
Politi-Psychotics' % of All False+POF Rulings	.	34%	.	64%	.	1.9%

Next up, an analysis of the “truthful” rulings.  On True-to-Half-True rulings, we also have some divergence, though not as great, and more to the favor of Republicans as well.  Ostermeier:

As a consequence, Democrats have therefore been presented as much  more  truthful  -  with over 75 percent of statements receiving the top three grades of True (16 percent), Mostly True (27 percent), or Half True (33 percent).

Less than half of Republican statements graded by PolitiFact were regarded as half truths or better - just 90 out of 191 (47 percent).

Category	Ostermeier Democrats	Politi-Psychotics Democrats	Ostermeier Republicans	Politi-Psychotics Republicans
True	16%	26%	(not known)	22%
Mostly True	27%	20%	(not known)	16%
Half True	33%	24%	(not known)	18%
Total	76%	70%	47%	56%

Ostermeier on "Barely True" rulings:

Republicans were also assigned a larger percentage of "Barely True" statements than Democrats, bringing the tally of all falsehoods or near falsehoods in the bottom three categories to 52.9 percent of Republican statements to just 24.6 percent of those made by Democrats.

While it’s true that the Republicans were assigned a larger percentage of “Barely True” statements than Democrats, my numbers show that overall the percentage was about 10 percent more in terms of the raw number of statements, with 84 “Barely Trues” for the Democrats and 93 for the Republicans. In percentage terms it looks closer:

Category	Democrats %age	Republican %age	Other %age
Politi-Psychotics Barely True as % of Total	13.2%	14.1%	16.1%
Politi-Psychotics Totals of False+POF+ Barely True as %age	30%	44%	35%
Ostermeier Totals of False+POF+ Barely True as %age	25%	53%	.

Finally, there’s the pundits and organizations outside the former and current office-holding politicians, including chain e-mails, bloggers, polls and bumper stickers, and in my count, celebrities and foreign officials and some odds and ends. The Total row includes the counts in parentheses. Ostermeier:

An examination of the more than 80 statements PolitiFact graded over the past 13 months by ideological groups and individuals who have not held elective office, conservatives only received slightly harsher ratings than liberals...Half of the statements made by conservatives received ratings of Pants on Fire (12.5 percent), False (16.1 percent), or Barely True (21.4 percent), compared to 41 percent for liberals.

Category	Ostermeier’s Percentage Conservatives	Politi-Psychotics' Percentage Conservatives	Ostermeier's Percentage Liberals	Politi-Psychotics' Percentage Liberal
Barely True	21.4%	18.1%	.	12.3%
False	16.1%	27.8%	.	19.6%
Pants on Fire	12.5%	17.2%	.	7.2%
Total	50.0%	63.1% (141)	41%	39.1% (54)
Office-Holders	53%	44%	25%	.

The office-holders’ percentage numbers for the bottom categories are included in the bottom row for comparative purposes. In all of my studies, the pundits and other groups are almost always ahead of the regular office holders in the “untruthful” categories, so Ostermeier’s finding seemed unusual. Further support to this “untruthful trend” by non-officeholders is lent by the consistent finding that those who have more comments evaluated tend to be more truthful (have a higher Politi-Score)…who are almost always office holders.

My next post will be about how I finally confirmed that Ostermeier used only PolitiFact National rulings in compiling his data, and how it may have raised an issue with regard to my other statistics.