The Rasmussen Way

Scott Rasmussen has become of the country's leading pollsters. When his firm Rasmussen Reports releases campaign numbers, many journalists treat them like holy scripture. Newsdesk is not the biggest fan of polling, so the opportunity to question him on his methods was unmissable.

The trigger was a Jan. 17 poll by Rasmussen Reporting that both Gov. Rick Perry and Sen. Kay Bailey Hutchison would pulverize Democrat Bill White in the race to be governor if the general election were held today.

Yet not all polls are created equal, so it pays to dismantle what's presented. First up, the methodology: 1,000 likely voters, contacted by phone, who responded to a recorded poll. Rasmussen said, "With a survey like this, we start by surveying a population of adults statewide. We use an automated survey system, and we establish the overall sample based on the adult sample. We then go through a screening process, first to ask them about their general participation in elections historically, and then their particular interest in the Republican primary elections coming up right now, and the people who survive the screening process go on to the rest of the survey."

But if this is a poll about the three-way Republican gubernatorial primary, the obvious question is, why lead off with a question about Bill White in the general election? Why no mention of the Democrats' perceived number two candidate, Farouk Shami? Why no Libertarian? After all, GOPer Debra Medina (prior to this poll, last seen languishing in single digits) gets on the question list. Rasmussen explained, "The primary interest that drove this survey was about the Republican primary, and when we spoke to a number of people about the prospects for the fall, it seemed like White was the best measure in terms of what we wanted to accomplish in terms of a general election comparison."

As for the screening process for who gets surveyed, Rasmussen said, "If we were just doing general election match-ups, we would do our sample, we would do a very light screen based on somebody's voting history, and then we would ask the general election questions. As we get closer and closer to elections, we begin to change our technique. We begin to ask more specific questions about the upcoming elections, we use a tighter screen based on people's response to that, and we also add supplemental questions about, how certain are you of your vote? Is it possible you might change your mind? If you were voting for a third party candidate, which way would you lean if you had to ask one of the two major party candidates? From Labor Day on, it becomes much more of a focused process."

So, on this lightly screened poll, four out of every ten respondents said they would favor White over Perry, right? Kinda. "We take our full sample, and we have a weighting process. The geography is handled through our calling system, and then we weight all of the results to match by gender, age, race and political party."

But it's not just raw (or weighted) numbers: The poll is accompanied by an analysis, containing such conclusions as "Like other Democrats around the country, White appears to be suffering from his party’s championing of the unpopular national health care plan." When asked to explain how that conclusion was jumped to, Rasmussen said, "We do national polling every single night, we do two separate tracking surveys, one on economic matters and one on political matters; we know from all the data we have collected that Democrats consistently are doing weaker among unaffiliated voters. They're all also struggling. The health care bill is remarkably unpopular for a signature piece of legislation."

It's therefore lucky that the word "appears" is in the analysis, because if the statement was more definitive, it would arguably be a logical fallacy, since no question was asked here about the specific reasons for voting for or against White. A plus B doesn't always equal C.

Considering this is supposed to be a poll about a Texas Republican gubernatorial primary, it's surprising how few questions are actually about Texas. Out of 14 top line results, five are about terrorism, two about the national economy, one about health care and one about President Barack Obama's performance. One Texas-specific question could raise a few eyebrows:

4* Governor Perry withdrew Texas from a federal program offering up to $700 million in education grants. He expressed concern about education guidelines from the federal government as a result of accepting the grant. Do you agree or disagree with Governor Perry’s decision to turn down $700 million in education grants.

Now, on the surface that seems very innocuous: But, and here's where the interpretation gets tricky, Perry "expressed concern." He didn't "say." He didn't "argue." He didn't "believe." And there was no counter-argument, even though many voices have backed U.S. Education Secretary Arne Duncan's statement that Perry's claim of a federal takeover of education is nonsense.

Rasmussen defended the phrasing, and said, "Questions like that, there is never a perfect way to ask that question. We were trying to get at something that has recently broken in the news, and if it stays in the news, we'll stay on top of it and we will find different ways to get at it."

Rasmussen has taken some serious hits over the last few years for using leading questions, whereby the phrasing or the order of questions are designed to create certain responses (like, say, loading a gubernatorial primary poll with questions that frame the election as a Texas vs. Washington death match.) One of his most controversial questions made it into this survey:

7* Should the December attempt to blow up an airliner as it was landing in Detroit be investigated by military authorities as a terrorist act or by civilian authorities as a criminal act?

As Eric Boehlert of Media Matters observed when Rasmussen asked the same question about the Fort Hood shootings, it's a misleading proposal, based on false assumptions, since terrorism is a criminal activity and is investigated by civilian authorities all the time. Rasmussen's general response to such concerns was simple: "For the most part, it's just people don't like a particular question, they get upset about it, and more generally they either don't pay attention to the details or they are just don't like the message that comes out of it."