Research map / Reinforcement learning / Visual and representation RL
A Unifying Normative Framework of Decision Confidence
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a normative "soft optimality" model of decision confidence (probability of being optimal, exponentiated reward under planning-as-inference) and validates it against perceptual and value-based psychophysics datasets using AIC comparisons.
Problem
Existing computational models of decision confidence apply only when options have equal reward, so they cannot explain how reward, priors and perception jointly shape confidence in decisions.
Approach
Defines decision confidence as the probability that a trajectory is optimal, set as the exponentiated reward of belief-action pairs; this maps to planning-as-inference / maximum-entropy RL and yields a confidence model that generalizes across tasks with asymmetric rewards and priors.
Key results
- On the perceptual task, soft optimality gave lower AIC than perception confidence for 5 of 10 subjects.
- On the value-based task, soft optimality had mean AIC 206.65 versus 282.11 for expected value ratio and was the best fit for 21 of 33 subjects.
- Soft optimality tracked the rate of confidence increase with value difference better than the expected value ratio.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Soft Optimality confidence instead of the Expected Value Ratio heuristic | Soft Optimality confidence model · Value-based decision-making task | AIC: 206.65 vs 282.11 | Helped |
| Use Soft Optimality confidence instead of perception confidence | Confidence models · Perceptual decision-making task with varying priors and rewards | AIC | Mixed |
| Use Soft Optimality confidence instead of the Expected Value Ratio heuristic | Confidence models · Perceptual decision-making task with varying priors and rewards | AIC | Mixed |
| Add a choice-bias parameter to the prior (ablation) | Confidence models · Perceptual decision-making task with varying priors and rewards | AIC | No clear effect |
| Add the parameter β to the optimality probability (ablation) | Soft Optimality · Experimental data fits | fit quality | No clear effect |
Surprising findings
- Half of the subjects in the perceptual task reported confidence in their choice rather than in their perception, as instructed, and the soft optimality model captured this.
Limitations
- Experiments involve only two choices and one action step, so the model's generalization to sequential decisions is untested.
- The exponential mapping from reward to optimality is justified by convenience and fit rather than uniquely derived.
- Fitted parameters and AIC differences are based on small subject samples and the authors caution against statistical reliability of some comparisons.
Experimental setup
Datasets: perceptual decision-making experiment with varying priors and rewards (10 subjects, 700 trials per session) value-based food decision experiment (33 subjects)
Benchmarks: perceptual decision-making confidence data (Gabor, asymmetric priors and rewards) value-based food decision confidence data
Metrics: AIC predicted rate of high-confidence reports
Baselines: perception confidence observation confidence expected value ratio
Builds on: POMDP planning as inference maximum entropy reinforcement learning Bayesian confidence hypothesis
Closest papers by meaning
- Human-Aligned Calibration for AI-Assisted Decision Making (NeurIPS 2023 · similarity 0.87)
- Learning to Decide with AI Assistance under Human-Alignment (NeurIPS 2026 · similarity 0.86)
- Preference Learning of Latent Decision Utilities with a Human-like Model of Preferential Choice (NeurIPS 2024 · similarity 0.84)
- Risk-Sensitive Theory of Mind: Coordinating with Agents of Unknown Bias using Cumulative Prospect Theory (ICML 2025 · similarity 0.84)
- Model Alignment as Prospect Theoretic Optimization (ICML 2024 · similarity 0.84)
- Towards Theoretical Understanding of Sequential Decision Making with Preference Feedback (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.