Saturday, November 21, 2009

Survey Design Bias

Another example of biased survey design (here is a link to the full survey results).
Ninety-four percent of Obama voters correctly identified Palin as the candidate with a pregnant teenage daughter, 86% correctly identified Palin as the candidate associated with a $150,000 wardrobe purchased by her political party, and 81% chose McCain as the candidate who was unable to identify the number of houses he owned. When asked which candidate said they could "see Russia from their house," 87% chose Palin, although the quote actually is attributed to Saturday Night Live's Tina Fey during her portrayal of Palin during the campaign. An answer of "none" or "Palin" was counted as a correct answer on the test, given that the statement was associated with a characterization of Palin.
Obama voters did not fare nearly as well overall when asked to answer questions about statements or stories associated with Obama or Biden -- 83% failed to correctly answer that Obama had won his first election by getting all of his opponents removed from the ballot, and 88% did not correctly associate Obama with his statement that his energy policies would likely bankrupt the coal industry. Most (56%) were also not able to correctly answer that Obama started his political career at the home of two former members of the Weather Underground.
I'm thinking if Zogby had asked questions about Obama that were of a similar nature to the questions about McCain/Palin, they would have gotten more of their negatives correct. For example, "which candidate has been tied to radical preacher Jeremiah Wright?" or "which candidate has a muslim half-brother?" probably would have gotten a higher response rate, and in similar proportion to the McCain questions. On the other hand asking about McCain, "which candidate said that auto industry jobs aren't coming back?" would have gotten a lower response rate.
I also wonder what the survey results look like controlling for education and income. Democratic voters tend to be poorer and less educated (and not because liberal policies are dumb - they are just more compassionate towards people who are less well-off). That didn't make the report either, but it was part of the survey.

No comments:

Post a Comment