Why Not Have “Repeated-Measures” or “Within-Respondent” Polling?
June 17, 2012 Leave a comment
If an alien species judged our intelligence based on public opinion polls, they would have no qualms about enslaving us and mining the planet for resources. Polling shows that people love deficit reduction, but hate everything that reduces the deficit. People have strong opinions on global warming, but not opinions that are strong enough to avoid being influenced by the current temperature. Now a new study finds evidence that views on inequality are influenced by whether or not we’ve been thinking about the concept of choice.
We found that when the concept of choice was highlighted, people were less disturbed by statistics demonstrating wealth inequality, less likely to believe that societal factors contribute to the success of the wealthy, less willing to endorse redistributing educational resources more equally between the rich and the poor, and less willing to endorse increasing taxes on the rich to help the country as a whole.
Pollsters are getting better at preventing the biases induced by the sequence and wording of questions, but they can’t keep up with the myriad of ways people can be influenced. This is troubling in a world where politicians and the media make a big deal about polling swings of just a few points.
I’ve often wondered about whether the future of polling will involve some kind of within-respondent measure of change. Right now we’re seeing this trend take hold in education through a renewed emphasis on student-level growth. Comparing the top-line numbers of different schools or cohorts is insufficient for dealing with individual differences and constantly shifting student populations. The only way to judge a teacher, school, or curriculum is to eliminate those confounds by looking at the individual progress made by specific students between Time A and Time B.
For polling, this would involve polling the same sample at Time A and Time B, rather than comparing two different samples. Doing so would greatly reduce the variability between samples and lessen the influence of psychological biases. Being able to say that support for gay marriage went up 4% within these 8,765 people is a lot more definitive than saying it went up 4% when compared to a previous measure from an unrelated sample.
Obviously whatever sample was selected for this within-respondent polling would have to be very large, and a good deal of work would have to be done to identify a group of people with the proper diversity, reliability, and lack of ulterior motives. This method wouldn’t be all that effective for short-term election polling, and there are also issues with the fact that the sample would not be particularly random. However, I think being able to see how policy views change within a specific group of people over time would add something that we don’t get under the current system of random sampling.
Ultimately I think this future of polling has two directions. One is an automated utopian future where the sample sizes involve a good chunk of the electorate and are so large there is almost no error. The other is finding ways to measure the opinions of people whose opinions at previous points in time are already known.
Savani, K. & Rattan, A. (2012). A Choice Mind-Set Increases the Acceptance and Maintenance of Wealth Inequality Psychological Science DOI: 10.1177/0956797611434540