Forecasting elections
A poll is not a forecast, and its margin of error does not cover the errors that actually decide elections. Here is what those numbers really say, and what has to be added to turn one into a probability.
What a poll measures
A poll measures a stated intention, at a moment, among a sample, weighted by an assumption about who will turn out. Every one of those four words is a source of distance from the result, and only one of them is covered by the published margin of error.
This is not a complaint about pollsters. It is the difference between measuring opinion now and forecasting a future vote: two different jobs, and the second one requires adding something to the first.
What the margin of error leaves out
The quoted margin — the familiar "± 3 points" — describes exactly one thing: the luck of having drawn this sample rather than another. It assumes a perfect draw and says nothing about anything else.
- The turnout model
- Someone has to decide which respondents will actually vote. That choice moves the number more than sampling does, and it is not in the margin.
- Non-response bias
- If one electorate answers surveys less willingly, no sample size fixes it. You simply reach more people from the same side.
- House effects
- Two serious pollsters, the same day, report different numbers because they weight differently. The spread between them is signal, not noise.
- Late movement
- Some voters decide in the final days, often after the last publishable poll.
- Subsamples
- The quoted margin applies to the whole sample. For one age band or one region it is far wider — often wide enough to make the figure unusable.
A lead inside the margin is not a tie
This is the commonest misreading there is. If a candidate leads by 2 with a margin of 3, coverage calls it "too close to call". That is wrong in both directions.
A two-point lead is still a lead: it makes that outcome more likely than the other, just not much more. Calling it a tie throws away real information; calling it a win invents information. The honest answer is a percentage between the two — something like 60 %.
Turning a lead into a probability
This is the step almost no coverage takes, and it is exactly the one that matters. It means deciding how large the plausible error is — not sampling error, total error, the kind you see by comparing final polls to past results.
And crucially: that error is correlated. When polls miss, they nearly all miss in the same direction, because they share the same turnout assumptions. This is why the same lead is not worth the same probability under every system: a two-round election, where the second round is an assumption about how votes transfer, carries uncertainty that an average of first-round polling does not contain.
What Predictly does with this
What it does
It works out which election you mean, gathers what is published — polling, analysis, official sources — drops repeats of the same wire story, weighs what argues each way, and answers with a probability carrying its confidence level and the documents it read.
Its limits here
- It does not aggregate polls itself or compute a weighted average: it reads what is published, including aggregations made by others.
- It does not model second-round vote transfers.
- For a distant or thinly covered election the confidence will be low, which is correct behaviour rather than a fault.
- It refuses an election already held, and a question whose contest or constituency is ambiguous.
The steps in detail are on the methodology. How to read the percentage you get: understanding probabilities.