Badly designed polls and surveys

A social network specialized in young people performed a few years ago a poll among its members to find out their habits of connection. By the way they asked a few questions to check their knowledge:

·         Do you know how much is a gigabyte of data? 63% answered yes.

·         Do you know how many photos fit in one gigabyte? It should be understood that the question refers to photos taken with a mobile phone, for otherwise the question does not make sense, as the answer depends on the dimensions and resolution of the photos. The authors of the poll assumed that a typical photo takes 70 kilobytes, which means that 14,900 of these photos would fit in a gigabyte (1,048,576 kilobytes). The correct answer they expected was “around 10,000.” (The other options offered were “around 5,000” “around 2,500” and “around 1,000”). Only 10% of those who took the poll gave the expected answer.

·         Do you know how many YouTube videos fit in a gigabyte? This question is clearly absurd, since the answer depends on the size of a YouTube video, which depends on its duration. The researchers estimated that an average YouTube video contains between 1.5 and 3 megabytes, so they expected “around 500” as the correct answer. (The other options were “around 200” “around 100” and “around 50”). Only 7% of respondents gave the expected answer.

From the previous answers, the pollsters concluded that young people do not know the size of a gigabyte of data, although they think they do. Apart from the fact that the survey is poorly designed, as the questions are ambiguous, to understand the results we need to know the answer to the following question:

·         Were the respondents told that a typical photo in the second question takes 70 kB, and that a typical video in the third question take about 2 MB? If they weren’t told, they were not given enough information to answer the questions, and the conclusion of the survey should be: young people don’t know the size of a typical photo or an average YouTube video. If they were given that information, the conclusion should be completely different: young people don’t know how to divide. In both cases, the conclusion drawn by the pollsters is wrong.

In fact, the correct response to such poorly designed questions should have been: I don't know. But today almost nobody gives that answer, because most of us believe that we know everything.


Thematic Thread on Statistics: Previous Next

Manuel Alfonseca