The Proof's In The Poll Results
Analysis By Scott Rasmussen
You know it’s a strange new world when Gary Langer, the director of polling at ABC, attacks a Democratic polling firm. By the way, the good folks at Public Policy Polling (PPP) took the attack in stride. The firm's Tom Jensen noted that “one of the most amusing things Langer and others in his cohort claim is that polls should not be judged by their accuracy.”
One might be tempted to blame the spat on the fact that Public Policy Polling found Fox News to be more trustworthy than ABC News. If that’s all it was, it could be written off as another episode of shooting the messenger like the effort that Rasmussen Reports recently endured.
But the real story has to do with the changing media landscape, most recently on display in the coverage of the Massachusetts special Senate race.
Rasmussen Reports is a new media outlet, digital from birth, and we informed our audience that this was a race worth watching two weeks before the stunning upset victory by Republican Scott Brown in an historically very Democratic state. Our first poll showed Brown within single digits and even closer among those most likely to vote. Our coverage was picked up by all sorts of new media sources and helped define the race for those in the Bay State. Looking back, The Politico’s Ben Smith wrote, “The overwhelming conventional wisdom in both parties … was that [Democrat] Martha Coakley was a lock. It's hard to recall a single poll changing the mood of a race quite that dramatically.”
ABC has an old-school media mentality and chose not to share the news of a close race with their audience. In fact those who rely on ABC News didn’t learn anything unusual was happening in Massachusetts until just four days before the election. By that time, Rasmussen Reports and PPP both showed the race to be a toss-up but trending toward Brown, and President Obama had decided to attend a campaign rally to help Coakley's floundering campaign.
The difference in coverage is that new media outlets are quite comfortable trusting people with information, while old media outlets view themselves as gatekeepers. The details of the gatekeeper mentality were outlined in Langer’s comments. The net result was that the old media's audience was ill-informed.
The reason is simple. Rasmussen Reports and Public Policy Polling use an automated polling system while ABC relies on an operator-assisted technology. According to the standards Langer set, automated polls “don't meet [ABC’s] standards for validity and reliability.” The frustrated Langer adds that he spends “a major chunk of my days locked in mortal combat” with such data.
Because of that view, ABC chose to rely on a Boston Globe poll, done with the same operator-assisted techniques that Langer prefers. That poll showed Coakley ahead by 17 points, so ABC concluded that there was nothing to report.
This was not an isolated incident. A similar phenomenon took place last year when automated polls in New Jersey accurately projected Republican Chris Christie winning, while operator-assisted polling generally showed incumbent Democratic Governor Jon Corzine ahead. That race prompted Democratic blogger Mickey Kaus to observe, “If you have a choice between Rasmussen and, say, the prestigious N.Y. Times, go with Rasmussen!”
Needless to say, the New York Times shares Langer’s views of automated polling. However, to their credit, even the Times acknowledged that some polls showed the race closer while quickly adding that those polls weren’t up to their standards. But at least they tried to explain why some people thought the race was getting closer.
Langer picks at some of the details in PPP polls that he doesn’t like. It’s easy to do that with any poll including the ABC data. Of the last dozen national job approval polls released, only two show the president above 50%. One of the two is an ABC poll. Does that make their data worthless? No, of course not. In fact, the ABC data is consistently among the most favorable to Obama, so informed readers can build that into their evaluation of the results produced by Langer and his team.
While picking at things he doesn’t like about PPP and automated polls, Langer also fails to note one of the obvious reasons that automated polls may catch surprise results that the operator-assisted technology may miss. In a place like Massachusetts, people might not want to admit to another human being that they are voting for a Republican (same thing in New Jersey last fall). They may be afraid that someone will try to talk them out of it or are just uncomfortable voicing it. But with an automated system, they can clearly state their views.
Automated systems also have advantages in terms of measuring trends. With the Rasmussen Reports system, a person being asked about the president’s job approval tonight is hearing exactly what someone heard a year ago. They are being asked exactly the same question by exactly the same person with exactly the same inflection, nuance, accent and timing.
This is not meant to disparage operator-assisted techniques that have been used since the time before answering machines were invented. They are useful, but the delivery is not as consistent for tracking purposes. Even the same operator may ask the question differently between the beginning of a shift and the end.
But the real discussion is not about the details of one polling approach versus another. It’s new media versus old media.
At Rasmussen Reports, we’re quite comfortable providing our audience with a tremendous amount of relevant and timely information. In fact, we’ll be adding new forms of data later this year to enhance our coverage of Election 2010. We are a source, not a gatekeeper. That’s why we post all the wording to our questions online. If you don’t like them or our interpretation, we give you the tools to make your own judgment.
We also post our track record because, unlike ABC’s Langer, we believe results matter. In our state polls for this election cycle, we are including references to our prior history in each state and access to a full list of our state polling results.
In 2008, Obama won 53%-46% nationally, and our final poll showed him winning 52% to 46%. While we were pleased with the final result, Rasmussen Reports was especially pleased with the stability of our results. On every single day for the last six weeks of the campaign, our daily tracking showed Obama with a stable and solid lead attracting more than 50% of the vote.
An analysis by Pollster.com partner Charles Franklin “found that despite identically sized three-day samples, the Rasmussen daily tracking poll is less variable than Gallup.” During Election 2008, the Rasmussen Reports daily Presidential Tracking Poll was the least volatile of all those tracking the race.
We also have provided a summary of our 2008 state-by-state presidential results for your review.
In 2004, George W. Bush received 50.7% of the vote while John Kerry earned 48.3%. Rasmussen Reports was the only firm to project both candidates’ totals within half a percentage point by projecting that Bush would win 50.2% to 48.5%. (see our 2004 state-by-state results).
Rasmussen Reports is a media company specializing in the collection, publication and distribution of public opinion information.
We conduct public opinion polls on a variety of topics to inform our audience on events in the news and other topics of interest. To ensure editorial control and independence, we pay for the polls ourselves and generate revenue through the sale of subscriptions, sponsorships, and advertising. Nightly polling on politics, business and lifestyle topics provides the content to update the Rasmussen Reports web site many times each day. If it's in the news, it's in our polls. Additionally, the data drives a daily update newsletter and various media outlets across the country.
Some information, including the Rasmussen Reports daily Presidential Tracking Poll and commentaries are available for free to the general public. Subscriptions are available for $4.95 a month or 34.95 a year that provide subscribers with exclusive access to more than 20 stories per week on upcoming elections, consumer confidence, and issues that affect us all. For those who are really into the numbers, Platinum Members can review demographic crosstabs and a full history of our data.
To learn more about our methodology, click here.