BROADCASTS.com

https://astralcodexten.substack.com/p/mantic-monday-scoring-rule-controversy

\xa0

Metaculus scoring rule controversy

Zvi considered using some Metaculus markets for his weekly coronavirus roundup, but was\xa0turned off by the scoring rules.

Ross\xa0Rheingans-Yoo\xa0writes about the issue here. Everyone agrees Metaculus\u2019 scoring rule is \u201cproper\u201d, a technical term meaning that it correctly incentivizes you to choose the probability you think is true. Zvi and Ross\u2019s objection is that it doesn\u2019t correctly incentivize you about whether to bet at all, or how much effort to put into betting.

For example, on many questions, you can make guaranteed-positive bets - you\u2019ll gain points on the prediction even if you were maximally wrong. If you were trying to maximize your Metaculus points, you would bet on all of these questions. If you were trying to maximize your Metaculus points in a limited amount of time, you might even bet on them without investigating at all. The person who spends one second picking a random number on a thousand questions will get more points than someone who spends an hour researching a really good answer to one question.