A blog entry posted at CQPolitics.com looked at a recent Rasmussen Reports poll and another by the Kaiser Family Foundation and concluded that the “new polls disagree on whether a government overhaul of the nation's health care system will leave people better off or worse off.”

In fact, according to the posting, the results were dramatically different with the Rasmussen poll showing that people think the system would be worse by a 54% to 20% margin “while the Kaiser poll says the country as a whole would do better, 54 percent to 27 percent.” The post concluded that the polls “do not provide much in the way of clues that might explain the discrepancy..”

Stories like these are frustrating and are used by some to raise doubts about the entire field of polling. At Rasmussen Reports, when we see a discrepancy like that, we seek to understand why.

In this case, at least, the problem has nothing to do with the polls and everything to do with the commentary. In fact, the Kaiser poll found almost identical results to the Rasmussen poll on similar questions.

For example, the Rasmussen Reports poll found that in late November 38% favored the plan working its way through Congress and 56% are opposed. At the same time, a majority of Americans say that major changes are needed in the health care system.

The Kaiser Foundation poll found that 35% want reform and like what they hear about the current proposals in Congress. Fifty-nine percent (59%) either don’t like the current proposals (33%) or don’t want Congress to pass health care reform at all.

The difference between 35% who like the current plans in the Kaiser poll is essentially the same as the 38% who favor it in the Rasmussen poll. So is the opposite—59% in the Kaiser poll and 56% in the Rasmussen poll. Both polls show a majority desire to pass some kind of reform.

So, how did the blog posting conclude that the results were so different? Because they compared a Kaiser question about health care reform in general to a Rasmussen question about the plan working its way through Congress. At a time when people want reform but don’t like what they’re hearing about the Congressional plan, that’s a pretty big difference. Compare apples to oranges and you make a mess.

For consumers of polling data, this is why it’s important to read carefully the questions that are asked. Even more important, it’s critical to see if those offering commentary and analysis are accurately reflecting what the polls say. In general, the weakest part of the polling process is the punditry commentary on it rather than the polls themselves. Many, perhaps most, of the time that people cite dramatically different poll results they are citing fundamentally different questions.

One general guideline is to place more confidence in the places where polls agree and urge more caution in places where they don’t. In health care polling, virtually all polls show that most Americans want reform but not the reform that is currently being considered in Congress. Other common ground can be found in the fact that Democrats consider it a top priority and like the Congressional plan. At the same time, Republicans and unaffiliated voters consider it a lower priority and don’t like the Congressional plan. Overall, the biggest concern that people with insurance have about reform is that they may be forced to change their existing coverage and all Americans are opposed to plans that would increase deficits or increase the cost of health care.