It's really a dim view of humanity to treat experts as black boxes who spit out predictions and can be judged by Brier Score ... And not even MENTION the possibility that one could ask people why they believe what they believe and try to figure out whether that reasoning is sound. :-P
What's the Brier score of Darwin? Or Einstein? If someone says AGI is impossible because silicon chips lack Godly souls, am I obliged to give them some deference if their Metaculus profile is sufficiently impressive?
I agree you can ask them, but you can do this for forecasters too, but this doesn't mean people don't accuse people of laundering vibes.
"How did you get from that explanation to 60%" feels pretty similar to "how did you get from that explanation to the sense that biden will probably stay". You've just pushed the vibes a level deeper.
It's really a dim view of humanity to treat experts as black boxes who spit out predictions and can be judged by Brier Score ... And not even MENTION the possibility that one could ask people why they believe what they believe and try to figure out whether that reasoning is sound. :-P
What's the Brier score of Darwin? Or Einstein? If someone says AGI is impossible because silicon chips lack Godly souls, am I obliged to give them some deference if their Metaculus profile is sufficiently impressive?
COI: I don't have a forecasting track record :-P
I agree you can ask them, but you can do this for forecasters too, but this doesn't mean people don't accuse people of laundering vibes.
"How did you get from that explanation to 60%" feels pretty similar to "how did you get from that explanation to the sense that biden will probably stay". You've just pushed the vibes a level deeper.