Tchotchkes: A GOP Halloween

Pants on Fire Factor: Obama-Fact

The addition of the state PolitiFacts began with the appearance of PolitiFact Florida in 2009, but really didn't really get in gear with publishing a substantive number of rulings until 2010.  In fact, in 2009, PolitiFact "Florida" published a grand total of 9 rulings, bringing in lead PolitiFact Florida writer Aaron Sharockman around October.  Then things took off in 2010 as Texas, Georgia, Rhode Island were added in the first half and then Ohio, Oregon, Wisconsin and Virginia in the second half.

 

It would seem as if the states came along to fill in the gap of the Congressional Quarterly, whose writers worked with PolitiFact doing rulings from 2007 through 2008.  However, as I described in my original assessment of this group and what its "affects" may have been , the types of statements selected at that time were of a different nature:  in the case of Congressional Quarterly, they related to its area of expertise, congressional positions and votes (although they did do other types).  In the case of the PolitiFact states, however, many rulings had to do with matters of a local nature, for example, one of the first PolitiFact Florida rulings on  a statement by then-Governor Charlie Crist who was bragging up the reduced rates of a negotiated, un-subsidized Florida healthcare program called Cover Florida Healthcare (found False).

 

The "Pants on Fire factor" (PoF) as I call it, a measure of what is claimed to be PolitiFact's bias for rating a False ruling as Pants on Fire, is shown below as aggregated from 2007 to September of 2012.    It is a comparison of Democrats to Republicans, with a factor of 1.0 translating to mean that PolitiFact assigns Pants on Fire to Democrats at the same (or equal) rate it assigns Republicans.  Anything below the 1.0 means Republicans are assigned Pants on Fire less than Democrats, anything above the 1.0 means Republicans are assigned Pants on Fire more than the Democrats (as a percentage of False + Pants on Fire rulings).

 

 

The red line shows the growth of the factor from favoring the Republicans to favoring the Democrats (or growing ever more negative for Republicans) over PolitiFact National's six years of Truth-o-Meter rulings.  The chart begins in quarterly increments until the end of 2008 to show how the Congressional Quarterly scored for that period as well, displayed in the green line. Thereafter the increments are annual.

 

At the end of 2008, PolitiFact National (including the Congressional Quarterly) had a factor below 1.0, although as shown by the green line, the Congressional Quarterly (CQ) alone was assigning Pants on Fire to Republicans 42% more than to Democrats.  With CQ behind them, in 2009 the PoF factor made its biggest jump ever at PolitiFact National, from .90 to 1.60.    For the year 2009 alone, PolitiFact National's Pants on Fire factor was a whopping 5.06.

 

Accumulated through the end of September of 2012, it now stands at 1.94--in other words, in the course of its history, PolitiFact National assigned Pants on Fire almost twice as often for Republicans as it did for Democrats.

 

Combined with the state PolitiFacts, however, the chart takes a different slope, as shown by the purple line.  It starts at the same high point as in 2009 because of there being only 9 PolitiFact Florida rulings, but as they accumulated, the aggregate Pants on Fire factor for PolitiFact as a whole dropped, from the 1.60 in 2009 down to its current 1.33.  The states themselves, if they were separated from PolitiFact National, would now average 1.21.

 

So it is indeed true that once PolitiFact "dissolved its partnership" with Congressional Quarterly, it began to appear as though it was increasingly assigning Falses as Pants on Fire to Republicans.  But working with CQ may not be a cause even with such a correlation, given that its own writers' assignment of Pants on Fire to Republicans were far more (58% more) than that of what PolitiFact National was doing at the end of 2008.  Were there other possible reasons?  What happened in 2009?

Announcing "Truth Index+"

While PolitiFact's Truth Index scoring methodology appropriately penalizes the lower scores, particularly Pants on Fire, it also distorts the overall score in a way that prevents valid comparisons from being made.   It basically takes comparisons out of proportion.   It is also more difficult to graphically present with its negatives.

 

Back in 2010, I began with a grading system called the Politi-Score, where a True was an A+ at 100, a Mostly True was a B at 90, down to a 50 for a flunking Pants on Fire.  That wasn't bad but it also made comparisons too "tight".  I then tried starting at 100 points working down to negative 25 for Pants on Fire, because others in a PolitiFact discussion group I presented my work in seemed to feel that the penalty was necessary as well.  About that time PolitiFact created its own measure called the Truth Index, which starts at 100 for True, makes Half True the zero and then goes down to negative 150 for Pants on Fire.  While I've stuck with that for a while now, I've realized in order to attempt things like standard deviations or z-scores, negatives might be difficult to work with.

 

This time I decided that True will be scored at 100 working down by equal 20 point increments to zero for Pants on Fire.  This will also make it easier for doing graphs by doing away with the negatives.  It also makes percentage comparisons easier to understand because the scores are perfectly proportional.  It is almost the equivalent of a percentage measure in and of itself.

 

So now a timeline can be developed from the beginning (2007) until the end of September of 2012.   I've also included the cumulative timeline for the Congressional Quarterly--the group that worked with PolitiFact when it first began in 2007, ending at the end of 2008.  This chart shows the Democrats Truth Index+ versus the Republicans, and the results aren't surprising considering everything else published on this blog.  The period measure is quarterly for the cumulative Truth Index until the end of 2008, to show the Congressional Quarterly Index, after which it is annual, so the chart would be less cluttered.

 

Click to enlarge: CQ rulings stop at end of 2008 when CQ-PolitiFact partnership ended

So today it can be said that based on 2,463 PolitiFact National rulings, it rates a Democrat about 29% more truthful on the Truth-o-Meter than a Republican.  And the largest drop in the Republicans Truth Index occurred from the end of 2008 until the end of 2009 (about 7%).  Whether it's because the Congressional Quarterly was no longer assisting is disputable--my feeling is it had something to do with the new Democrat president's term and the advent of healthcare reform in congress.

 

Next up will be a look at the "Pants on Fire" factor for the same data.

PolitiFact Rulings thru September, 2012

Reviewing the PolitiFact rulings almost one by one from since "in the beginning" to re-verify my totals and make the details more accurate has been quite the experience.  It's taken almost two months, and I found I made a lot of mistakes:  not enough to change all the averages I've presented in this blog, except in one area, which is the "Official/Non-Official."  I believe the well known  Eric Ostermeier study of 511 PolitiFact rulings from 2010 used those currently or previously in an elected office: I decided that was not a good way to do it because there may be a difference in the level of truthfulness between being in office and not being in office.  Office-holders were only designated as such when they were in an elected office at the time the statement was made as evaluated by PolitiFact.

 

I had a couple gender and party misidentifications,  plus quite a few records which were done by the state PolitiFacts which I had included with National for through 2010.  My ruling count now corresponds with what someone at PolitiFact kindly told me their number was.  So, through the end of September, 2012,  PolitiFact added a record number 556 rulings for the quarter, to now total 6,274 rulings.  Here is how they breakdown affiliation-wise:

 

 

There are a lot of reasons why PolitiFact has selected proportionately more Republican statements than Democrats, the main one being that outside the president, Republicans have dominated state/local governments, particularly the PolitiFact regions that started up in 2010, such as Ohio (John Kasich), Florida (Rick Scott), Virginia (Bob McDonnell), Texas (Rick Perry), and Wisconsin (Scott Walker).  If PolitiFact had a New York or California affiliate, I'm guessing we'd see more Democrat rulings.

 

Next the revised Truth Index--which I'm calling Truth Index+ for now-- will be explained featuring the new database.

Tchotchkes: Post Rape Nuts

Sidebar: Official Errors

Currently (as I've posted about before) I am doing a very tedious, time consuming task with regard to past PolitiFact rulings (about 4,000 of them from 2007 through 2011):  I am refining my "rough" database to include specific dates, statement descriptions and claimant's (current) position.  When done, the database will be much more accurate than before with regard to these details, and hopefully will provide some greater insight into statistics regarding PolitiFact's Truth Index, as well as more reference material for general posts.

 

My biggest errors thus far have not been with the Truth-o-Meter rulings themselves, it has been with the "claimant position."  This was added when the scholarly Eric Ostermeier study of 500 rulings in early 2011 came out:  he had separated those in an official position with those who were not.  To some "not an elected official" might mean the party boosters, the media pundits, the organizations.  This was basically how I quickly classified them when I thought I'd try to replicate Ostermeier's results.  But there are many individuals who were not elected officials when they sought office, or were in a different office.  For example, before running for governor of Florida, Rick Scott was a businessman. Senator Marco Rubio was not in any official position when he ran for senator, although he had been prominent in state government.  The way these two should be recorded, up until they assume office, is not as an official.  In other words, a PolitiFact "rulee's" position should be recorded as the one they were in at the time they made the statement.   In the Scott/Rubio examples, I had them recorded (mistakenly) as officials up through 2010.  While most strictly Democrat-Republican Truth Index comparisons would hold, anything with regard to official/non-official would be suspect, because Scott and Rubio comprised a large number of rulings (as well as others).

 

Other than that, when complete, there should be new, very accurate chart data to report.   I am also considering a new statistically "relative" approach other than PolitiFact's own Truth Index and the Politi-Score, which I will be testing with the new "cleaned up" database.  In the mean time, thank you for your patience!

Sidebar: Does Ze Bra Hold Up?

I knew it might be too good to be true when my conservative counterpart appeared to stop posting his criticisms of PolitiFact at both of his sites...as well as stop posting at the PolitiFact Facebook page.  I also thought he might be up to something.  In my last contact on the PolitiFact Facebook page, he said he was going to do the same Pants on Fire analysis of the state PolitiFacts, and it seemed likely to me he was going through the process of collecting the state data which was taking up his time.

 

It also seemed he was getting heard at a higher level in the right-wing media where he was cited and featured with an interview; he had also "un-blogspotted" his PolitiFact Bias site, making it a real website, in order to give it more "cred" I suppose.  There's most likely money to be made in connections to Big Government or Weekly Standard especially when you're regularly trashing an "independent fact-checker" as being liberally biased.  Now that he was being cited by the bigger names, it seemed that would be a natural progression.

 

As it turns out, author Bryan White has decided to launch a new website where he does his own fact-checks with a conservative bent.  Of course, he's not rating them like PolitiFact or the Washington Post Factchecker, and he's given the site the name of Zebra FactCheck, which I think reflects his ideology very neatly--for most conservatives don't see the world in shades of grey, they see it in black and white.  This also is very indicative of  the false dichotomies he often presents.  But in terms of the right-wing websites, he may have segued into something that might get him a real job.

 

Speaking of that, this makes my theory seem all the more likely that Bryan White may have been someone rejected for a journalist-type job by the Tampa Bay Times or PolitiFact itself: he states that these fact-checks are what he does best.  So this bolsters my theory that this was his retaliation.  But that's pure speculation.

 

All in all, it makes it easier for me in the sense that it is harder to make a criticism of a criticism of a factcheck than it is just to criticize the factcheck itself.  And that I will do.  In fact, I've discovered that the best way to check his fact-checks may be with PolitiFact's fact-checks.

 

In the meantime, I am having fun trying to find a comical symbol for "Checking Zebra-Factcheck".  I thought of De-Zebtions and the graphic for the title of my post "does Ze-Bra hold up?'  Or there's the black and white angle.  Lil Black and White Lies?  "Gray"-ding Zebra Factchecks" maybe?   Stay tuned.

Truth Index: Obama-Romney One Month Out

Dreaming of a Michele anniversary gift

Recently I posted about Mitt Romney's 47% remarks, saying it might be the game changer and even though it was rated "True" by PolitiFact, it might be considered a candidate for Lie of the Year...unless, as I said "...Obama could make his own big gaffe" which would reverse Romney's course following that statement. Well did he ever. I got up Thursday morning hoping the first debate had just been a bad dream. But I did enjoy SNL's take on it, which was just about the whole show.

 

But life must go on and it's time again to look at the Truth Index trend line for Romney and Obama. It hasn't changed much (unlike the polls), so I thought I might add something to show what appears to be the "Truth Index trend" for presidential candidates during their campaigns, i.e., the Truth Index of 2008 Republican presidential candidate John McCain. We see how all three candidates had slipping Truth Indexes as their campaigns went on. In my last post on this, and as can be seen in the chart below, Romney had a higher Truth Index than Obama until after he announced his candidacy in 2011, after which it deteriorated substantially.

Click to enlarge: All 3 candidates' Truth Index fell off during the campaign period

This chart represents 427 rulings for Obama (which may explain why his Truth Index has not fallen as much in 2012), 183 for Romney, and 169 for McCain, from about June of 2007 to present. Today Obama's Truth Index has bumped down a bit to a positive 11.48, and Romney is at a negative 16.12. John McCain's Truth Index, most of his rulings from the 2007-8 period, is a negative 8.88, roughly half that of Romney based on a similar quantity of rulings. For what it's worth, looking at the infamous "Pants on Fire" factor on an individual basis, Pants on Fire were assigned nearly three times as much to Romney than Obama (PoF 3.86), and twice as much to McCain than Obama (PoF 1.97).

Truth Index: CQ Tips

When PolitiFact first began its operations in 2007, it was a "joint venture" with the Congressional Quarterly (CQ), which until 2009, was owned by the same company as the Tampa Bay Times. This may be the reason why, at the end of 2008, the partnership with CQ was dissolved.

 

A claim by PolitiFact Bias in its "research" necessitated my review and update of 2007-10 statistics collected (which I had not dated or listed the writer) to determine what the CQ "input" might be:

When PolitiFact first launched, it existed as a joint venture between the St. Petersburg Times newspaper and the Congressional Quarterly magazine, both owned at the time by the Poynter Institute in St. Petersburg, Florida. The numbers for 2007 and to a lesser extent for 2008 demonstrate tendencies not present in later years. Some of the differences may stem from the vigorous Democratic primary season or from the removal of input from CQ staff when the partnership was dissolved after the 2008 presidential election. Simply put, PolitiFact’s bias against Republicans as measured by its use of the “Pants on Fire” rating has intensified since the separation from CQ. Republicans during the more recent era receive a “Pants on Fire” rating 143% more often than Democrats (see Fig. 2).

At this point I have reviewed enough data to have completely recorded all rulings for 2007-08, the period of time that CQ participated. CQ has a large staff and spread out PolitiFact rulings among many of their writers and researchers, but because the Congressional Quarterly itself was considered the primary publication for the CQ staff, it would appear that there were a only a few people assigned to PolitiFact full time. That being said, during the time they worked with PolitiFact, they published 170 rulings out of about 800 total, about 20 percent.

 

A point needs to be made with regard to the rulings in that, because the Congressional Quarterly's area of expertise seemed to be what was happening in Congress, the majority of its rulings appear to relate to that as shown in the graphic below on Joe Biden: votes and positions in Congress. This may serve to also assure that "Pants on Fire" is not as much an option.

Click to enlarge:  These are all rulings authored by CQ staff members--"bills".

So let's take a look at the period from the third quarter of 2007, about when PolitiFact began to get established and enough rulings accumulated, and compare its Truth Index without the Congressional Quarterly rulings to that of just the Congressional Quarterly rulings. What we might expect is more "neutrality" to show--perhaps even a Truth Index favoring the Republicans, or at least very even, along with the crucial Pants on Fire indicator designed by PolitiFact Bias, to be around 1.0, or I'd say at least 5 percentage points within it, meaning a fair assignment of Pants on Fire. As I finish the dating for 2009 and 2010 data (I am 80% complete at this point) a chart can be built to see just how the Pants on Fire have "intensified" on a timeline.

 

The accumulated timeline for Democrats and Republicans is shown below, with the CQ Truth Index showing in the dashed lines. In the beginning, it seems we get a hint of PolitiFact's so-called liberal bias, with a 44 Democrat Truth Index, although the number of rulings is not significant enough--CQ at this point had only contributed 21. By the end of 2007, however, the Truth Indexes converged a bit, with the CQ Republican Index in the lead for "truthiness."

 

By the second quarter of 2008 the CQ Democratic Truth Index surpassed that of the Republicans, ending at a positive 9.1 for the Democrats and a positive 4.6 for the Republicans. The PolitiFact-team -only rulings, however, showed the Democrat-favorable variance it shows to this day at negative 11.9 for the Republicans and positive 13.6 Democrats. For all PolitiFact rulings accumulated through the first half of 2012, this compares today to a near-zero +.3 overall for the Democrats and a negative 23.5 for the Republicans. In other words, while the Truth Index still favors the Democrats today, both parties have been eroding steadily since 2007.

Click to enlarge:  This is a graph showing the Truth Index as accumulated.

Chain e-mails.  The last column of the chart is titled "No chains" because of the large number of chain e-mails PolitiFact started investigating and reporting on. In this last column the Truth Index is adjusted to show what it would be without those rulings. Before I had separated the e-mails for affiliation based on a cursory review of the ruling; this time I made it much more than cursory to try to make the affiliation as accurate as possible.

Tchotchkes: Obama Needs an "Anger Translator"

Truth Index: Yes I know, Get A Life!

It was late in 2010 when I first thought about doing my own database of PolitiFact rulings.  So between Christmas and New Years of that year, when there was a brief stand-still in publishing due to the holidays, I went through PolitiFact's personality page, linking to each person or group on the list and then recording the count.  So each record reflected the name and political affiliation, along with a total by True, Mostly True, etc.

 

In 2011, I started a more detailed analysis.  Each record was one ruling:  I also included the date of the ruling, subject and writer.  Late in 2011, I decided to include a brief description of the ruling.

 

In doing so I realized an historical account of PolitiFact was needed--the "up to 2010" rulings should be recorded in the same way, as one by one rulings instead of by person/group.  Recently my conservative counterpart made allegations with regard to Congressional Quarterly (CQ), which was PolitiFact's partner in 2007 and 2008.  The only way I could test their theory--that CQ mitigated PolitiFact's alleged left wing bias--was to record those rulings in this way, so that a complete timeline could be created, and the CQ rulings could be filtered out for analysis.

 

Luckily, PolitiFact doesn't delete "old" rulings.  So I've been spending a lot of time in a somewhat tedious process, although it's been interesting reading some of the claims made back then, as well as seeing how PolitiFact grew and progressed as a fact-checker, which I will post about soon.

 

Between other things going on in September (personally) and doing this, I've not done a whole lot of posting.  It should be picking up soon, however, as we look at PolitiFact with an historical perspective.