When to Trust a Decision Model
A confidence number is only worth what it lets you trust. I put nine decision models, open and hosted, through one test: keep only the answers each was 90% sure of, and score them, first on 111 hard AI-written questions and then on 1,599 questions labelled by 100 people each. Some earned the trust. Here is which, and under what conditions.
- Decision Models
- Calibration
- Evaluation
- +2 more