The Gallup poll on the legalization of marijuana

Recently Gallup published a poll, Record-High 50% of Americans Favor Legalizing Marijuana Use, claiming, as the title indicates that 50% of Americans favour legalization of marijuana.

Jose Paulo Carneiro Ph.D., a Brazilian Professor of mathematics says that “it is very surprising that an Institute so renowned as Gallup, in a country so developed in matters of survey and research, makes a telephone survey and draws a conclusion about the opinion of “half of the Americans”.

Professor Jose Paulo Careiro states:

First of all, what is the methodology of this survey? What is the universe? Of course, it cannot be all the Americans.

Suppose it is the American population between 15 and 64 years (approximately 205.7 million).

If the sample were a simple random sample (that is, all the individuals have the same probability of being selected), the fact that the sample size is a tiny proportion of the population would not matter, because in the formula for the sample standard deviation (let us call it s) of a proportion to be estimated, the size N of the universe only appears in a factor, which, for a sample size of 1,005, equals 0.999995, which is practically 1.

Then, the maximum value for s, which occurs precisely for the 50% proportion and is independent of N, reduces to 1.6%. This means (supposing, as usual, the normality of the sampling distribution) that there is a probability of 95% that a real proportion of 50% will appear in the sample between 46.8% and 53.2% (this is the meaning of the phrase “the survey error is 3.2%”), which is a very acceptable value.

The problem is not in the sample size. The problem is that a telephone survey is not a simple sample survey, because not all individuals have the same chance of being selected. If you don’t have a telephone number, your probability of being selected is zero. If you have three telephone numbers and your neighbor has only one, your probability of being selected is three times his. Moreover, even inside a specific household, the probabilities are different. In certain households (mine, for instance), the probability that the husband answers the phone is very small compared with the probability that the wife does it. And, what is worse: the sample is biased, because there may be – and usually there is – a specific profile of people who answer, opposed to that of people who don’t answer the call.

In summary, it is very surprising that an Institute so renowned as Gallup, in a country so developed in matters of survey and research, makes a telephone survey and draws a conclusion about the opinion of “half of the Americans”.