Badly designed polls and surveys

A social network specialized in young people performed a few years ago a poll among its members to find out their habits of connection. By the way they asked a few questions to check their knowledge:

·         Do you know how much is a gigabyte of data? 63% answered yes.

·         Do you know how many photos fit in one gigabyte? It should be understood that the question refers to photos taken with a mobile phone, for otherwise the question does not make sense, as the answer depends on the dimensions and resolution of the photos. The authors of the poll assumed that a typical photo takes 70 kilobytes, which means that 14,900 of these photos would fit in a gigabyte (1,048,576 kilobytes). The correct answer they expected was “around 10,000.” (The other options offered were “around 5,000” “around 2,500” and “around 1,000”). Only 10% of those who took the poll gave the expected answer.

·         Do you know how many YouTube videos fit in a gigabyte? This question is clearly absurd, since the answer depends on the size of a YouTube video, which depends on its duration. The researchers estimated that an average YouTube video contains between 1.5 and 3 megabytes, so they expected “around 500” as the correct answer. (The other options were “around 200” “around 100” and “around 50”). Only 7% of respondents gave the expected answer.

From the previous answers, the pollsters concluded that young people do not know the size of a gigabyte of data, although they think they do. Apart from the fact that the survey is poorly designed, as the questions are ambiguous, to understand the results we need to know the answer to the following question:

·         Were the respondents told that a typical photo in the second question takes 70 kB, and that a typical video in the third question take about 2 MB? If they weren’t told, they were not given enough information to answer the questions, and the conclusion of the survey should be: young people don’t know the size of a typical photo or an average YouTube video. If they were given that information, the conclusion should be completely different: young people don’t know how to divide. In both cases, the conclusion drawn by the pollsters is wrong.

In fact, the correct response to such poorly designed questions should have been: I don't know. But today almost nobody gives that answer, because most of us believe that we know everything.


Thematic Thread on Statistics: Previous Next

Manuel Alfonseca

Scientific facts, historical facts

The possibility of repeating an experiment is one of the fundamental principles of the scientific method. No discovery is considered final until it has been confirmed by an independent team or researcher. If this happens, it becomes part of the scientific heritage. It follows, therefore, that a fact can only be considered scientific if it can be reproduced.

Historical facts are treated in a very different way. Documents describing the event are sought to confirm that it actually happened. Those documents are analyzed to estimate their degree of credibility. A historical fact will be more credible as a function of the number of independent documents that tell about it. The assassination of Julius Caesar is a well-documented historical fact, but it is not scientific, as it cannot be reproduced.

The origin of life is an event that most likely happened only once in Earth's history. It is impossible to repeat it, so as to study how it happened, therefore it is not a scientific fact. It is a historical fact.

What would be the documents, in the case of the origin of life? Fossil remains. But it is practically impossible to find them. Therefore, the origin of life will most likely be an insoluble problem forever.

But what if one day we make synthetic life in the laboratory? Wouldn’t we know then how the origin of life took place? Well no, for we wouldn’t be sure that the way in which we had made synthetic life were the same as when it appeared spontaneously, a few hundred million years after the origin of the Earth.

The same post in Spanish

Thematic thread on primitive life: Previous Next

Thematic thread on synthetic and artificial life: Previous Next

Manuel Alfonseca