The margin of error is *the* most widely misunderstood and misleading concept in

statistics. It’s positively frightening to people who actually understand what it means to see

how it’s commonly used in the media, in conversation, sometimes even by other scientists!

The basic idea of it is very simple. Most of the time when we’re doing statistics, we’re doing statistics based on a sample – that is, the entire population we’re interested in is difficult to study; so what we try to do is pick a *representative subset* called a sample. If the subset is *truly* representative, then the statistics you generate using information gathered from the sample will be *the same* as information gathered from the population as a whole.

But life is never simple. We *never* have perfectly representative samples; in fact, it’s

*impossible* to select a perfectly representative sample. So we do our best to pick good

samples, and we use probability theory to work out a predication of how confident we can be that the

statistics from our sample are representative of the entire population. That’s basically what the

margin of error represents: how well we *think* that the selected sample will allow us to

predict things about the entire population.

The way that we compute a margin of error consists of a couple of factors:

- The
*size*of the sample. - Magnitude of
*known*problems in the sample. - The
*extremity*of the statistic.

Let’s look at each of those in a bit more detail to understand what they mean, and then I’ll explain how we use them to compute a margin of error.

The larger a sample is, the more likely it is to be representative. The intuition behind that is

pretty simple: the more individuals that we include in the sample, the less likely we are to

accidentally omit some group or some trend that exists within the population. Taking a presidential

election as an example: if you polled 10 randomly selected people in Manhattan, you’d probably get

mostly democrats, and a few republicans. But Manhattan actually has a fairly sizeable group of people

who vote for independents, like the green party. If you sampled 100 people, you’d probably get at

least three or four greens. With the smaller sample size, you’d wind up with statistics that *overstated* the number of democrats in Manhattan, because the green voters, who tend to be very liberal, would probably be “hidden” inside of the democratic stat. If you sampled 1000 people, you’d be more likely to get a really good picture of NYC: you’d get the democrats and republicans, the conservative party, the working families party, and so on – all groups that you’d miss in the

smallest sample.

When we start to look at a statistic, we start with an *expectation*: a very rough sense

of what the outcome is likely to be. (Given a complete unknown, we generally start with the Bayesian

assumption that in the presence of zero knowledge, you can generally assign a 50/50 split as an

initial guess about the division of any population into exactly two categories.) When we work with a

sample, we tend to be *less* confident about how representative that sample is the farther the

measured statistic varies from the *expected* value.

Finally, sometimes we *know* that the mechanism we use for our sample is imperfect – that

is, we know that our sample contains an unavoidable bias. In that case, we expand the margin of error

to try to represent the reduced certainty caused by the known bias. For example, in elections, we

know that *in general*, there are certain groups of people who simply *are less likely*

to participate in exit polls. An exit poll simple *cannot* generate an unbiased sample, because the outcome is partially determined by *who* is willing to stop and take the poll. Another example is in polls involving things like sexuality, where because of social factors, people are *less likely* to admit to certain things. If you’re trying to measure something like “What percentage of people have had extramarital affairs?”, you *know* that many people are not going to tell the truth – so your result will include an expected bias.

Given those, how do we compute the margin of error? It depends a bit on how you’re measuring. The

easiest (and most common) case is a *percentage* based statistic, so that’s what we’ll stick

with for this article. The margin of error is computed from the *standard error*, which is in

turn derived from an *approximation* of the standard deviation. Given a population of size P;

and a measured statistic of X (where X is in decimal form – so 50% means X=0.5), the standard error E

is: