How Prolepsia’s public forecasts have scored.
This page scores every public question Prolepsia forecast that has since resolved. It reads the forecasts and the outcomes from the app each time the page loads, so it changes as questions resolve.
Each question gets a Brier score. Lower is better.
Overall
- Questions scored
- 12
- Average score, yes/no questions
- 0.146 (12)
- Average score, multiple choice questions
- None yet
- Resolved public questions
- 12
- Listed without a score
- 0
- Left out
- 0
For reference: always answering 50% on a yes/no question scores 0.25, whatever happens.
Scored questions
How the score works
The Brier score measures how far a probability was from what happened. On a yes/no question it is (p − o)², where p is the chance given to yes and o is 1 if the answer was yes and 0 if it was no. It runs from 0, a certain forecast that was right, to 1, a certain forecast that was wrong.
On a multiple choice question it adds up (p − o)² over every option, where o is 1 for the option that happened and 0 for the others. It runs from 0 to 2.
Only forecasts made before a question’s cutoff count. The cutoff is the earlier of the question’s resolution date and the date it resolved, from the start of that day. A forecast made on that day or later is left out, because by then the outcome may have been known.
A question’s score is the average Brier score of every forecast counted, each weighted equally. The overall figure for each question type is the average of the question scores.
The table also shows the last forecast before the cutoff and the chance it gave to what happened. That number alone would flatter the record: it can be made close to the outcome, when the answer is easier to see.
Number, date and count questions are listed with their outcome but no score. The Brier score needs a probability for each possible answer, and these questions forecast a range instead.
Questions that were annulled or cancelled are left out and counted, and so are questions with no forecast before the cutoff. A resolved question whose outcome does not clearly name one answer is listed without a score, with the reason, rather than guessed.