How Not to Design a Survey

Roy Morgan, I will not be completing your survey. Are companies actually paying money for your survey results? The survey is poorly designed and it is hard to imagine you are gathering useful and accurate information from a relevant cross-section of the population.

Let’s start with the obvious; the survey is two booklets totalling 163 pages. Who has the time? My compensation for completing the survey is a $20 petrol voucher. I do not currently own a car. One booklet has to be completed every day for a week. It will take me hours to complete.

The survey is two booklets totalling 163 pages. Who has the time to complete it?

Second, the questions are out of date. Your survey asks if I play video games. I do enjoy a round of Angry Birds from time-to-time on my mobile phone. My friends play games on their iPads or Facebook. But my choices for video games did not include internet games. So I had to answer that I do not play. I do not have any friends that play on traditional consoles but almost all my friends play games on the internet. The Economist had a special report about video games. Did you know gaming is the fastest growing media industry? But the average gamer is older and plays games on the internet, not a traditional console. All of those people are forced to say they do not play games according to the choices on your survey.

The survey needs modernising. It asks about video games but doesn’t include games played on mobile phones or tablets.

Finally, the questions are confusing and poorly laid out. The time frames switch in the same questions. In the same table I have to answer if I have “eaten-in, had take-away or purchased” food from various establishments in the past SIX MONTHS, then I have to say the number of times I visited in the past FOUR WEEKS and the amount I spent in the past SEVEN DAYS. Your questions are filled with underlining and bold type. I find the layout very distracting.

Here are my suggestions. 1) Modernise your survey. The questions need updating and the surveys should be offered online. 2) Offer short but more frequent surveys. I would be willing to do a 10-15 minute survey every month. I could sign up to receive the survey and complete it online.  3) Better incentives to complete the survey. If you want me to even consider completing your survey, you need to offer a bigger incentive, closer to $150. Or instead of a $20 petrol voucher, you could donate $5 to my favourite charity every time I complete a short monthly survey.  I have more tips but if you start with those you will be on the right path. I know survey design is hard but a poorly designed survey is worse than no survey.

This entry was posted in Uncategorized. Bookmark the permalink.
  • Jeff Ballweg

    Not to mention the hundreds and hundreds of tickboxes asking if I’ve heard of every different car sold in NZ, then hundreds more asking if I’ve seen an advert for those same models. Ridiculous.