Forecasting film awards
An Oscar is not an aesthetic judgement, it is the result of a vote. And several other votes, by people who overlap with that electorate, happen first. That is what makes it forecastable — up to a point.
What is actually being forecast
You are not forecasting which film is best. You are forecasting what an electorate will do: a few thousand members of a professional academy, with their crafts, their habits and their loyalties. "Which film deserves to win" and "which film will win" are barely the same question.
Precursors, and what each is worth
A precursor is worth exactly as much as the overlap between its electorate and the one you care about. That is the only criterion, and it explains the whole ranking below.
- The guild awards
- Producers, directors, actors and writers vote in their own guild before voting at the Academy, and they are largely the same people. That is why these predict best. The producers' award is watched especially closely because it uses a preferential ballot, as Best Picture does.
- Foreign academies
- A neighbouring but distinct electorate, with its own national tastes. Useful, less decisive.
- Critics' awards
- They predict what critics think extremely well, and what an academy of working professionals will vote for much less well. Confusing the two is the commonest mistake in this subject.
- Box office and audience scores
- Almost unrelated. A hit can be ignored; a film almost nobody saw can win.
The ballot changes the answer
Best Picture is decided by a preferential ballot: each voter ranks the nominees, the lowest is eliminated and its votes redistributed, until one film passes half. The other categories are decided by simple plurality.
The consequence is counter-intuitive and decisive: a divisive film placed first by many voters can lose to a film almost nobody ranks first but almost everybody ranks second or third. A preferential ballot rewards the absence of rejection as much as enthusiasm.
When the pattern breaks
Precursors do get it wrong, and they get it wrong in a recognisable way: when they disagree with each other. While producers, directors and actors all point at the same film, the rest is fairly legible. The moment they split across two or three titles, the front-runner's probability should drop sharply — and that is usually when coverage starts asserting a certainty the evidence no longer contains.
A good forecast here therefore rarely looks like 90 %. It looks like "55 % for this one, 30 % for that one, and the rest elsewhere".
What Predictly can do with it
What it does
It works out which ceremony and category you mean, researches what has already been handed out and what is being written now, drops repeats of the same article, weighs what argues each way, and answers with a probability carrying its confidence level and the sources it read.
Its limits here
- It does not know the voting calendar: ask before a single precursor has landed and the confidence will be low, which is the correct behaviour.
- It does not model preferential redistribution. It reads people who do take it into account.
- A campaign also turns on things that are never published. Some of the uncertainty is irreducible.
- It refuses a ceremony already held, and a category whose name is ambiguous.
The calculation in detail is on the methodology, and how to read the percentage is on understanding probabilities.