Research map / Probabilistic and scientific modeling / Molecules and proteins
Preference Learning of Latent Decision Utilities with a Human-like Model of Preferential Choice
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds a tractable neural surrogate (CRCS, and LC-CRCS with cross-feature linear context) of a computationally rational context-dependent choice model to improve preference learning from human choices.
Problem
Preference learning relies on Bradley-Terry-style choice models that ignore context effects in human choice; the cognitive models that capture these effects are intractable for inference.
Approach
Replace intractable computations of a computationally rational choice model (Howes et al.) with neural surrogates trained to predict expected utilities and choice probabilities, generalize it to a space of utility functions, and add a linear cross-feature context mechanism from LCL (LC-CRCS).
Key results
- LC-CRCS achieves the lowest summed NLL on Hotels, District-Smart and Dumbalska versus Bradley-Terry, Bower & Balzano and LCL (p<0.01), while LCL is best on Car-Alt.
- On District-Smart rank consistency (Kendall's tau), CRCS reaches 0.622 and LC-CRCS 0.525 versus 0.286 for LCL and 0.162 for Bradley-Terry.
- In Dumbalska elicitation, CRCS variants make significantly better expected-likelihood predictions than baselines at nearly all time steps with few queries.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CRCS instead of Bradley-Terry | Human choice prediction · Hotels | NLL: 573 -> 536 | Helped |
| Add LCL-style cross-feature context effects to CRCS (ablation) | CRCS · Hotels | NLL: 536 -> 536 | No clear effect |
| Use LC-CRCS instead of LCL | Human choice prediction · District-Smart | NLL: 3305 -> 3276 | Helped |
| Add LCL-style cross-feature context effects to CRCS (ablation) | CRCS · District-Smart | NLL: 3371 -> 3276 | Helped |
| Use CRCS instead of LCL | Human choice prediction · Car-Alt | NLL: 7290 -> 7322 | Hurt |
| Add LCL-style cross-feature context effects to CRCS (ablation) | CRCS · Car-Alt | NLL: 7322 -> 7345 | Hurt |
| Use LC-CRCS instead of Bradley-Terry | Human choice prediction · Dumbalska | NLL: 103669 -> 99147 | Helped |
| Add LCL-style cross-feature context effects to CRCS (ablation) | CRCS · Dumbalska | NLL: 100450 -> 99147 | Helped |
| Use CRCS instead of Bradley-Terry | District-Smart utility inference · District-Smart | Kendall's τ rank consistency: 0.162 -> 0.622 | Helped |
| Add LCL-style cross-feature context effects to CRCS (ablation) | CRCS · District-Smart | Kendall's τ rank consistency: 0.622 -> 0.525 | Hurt |
| Use CRCS instead of the original Howes et al. model for utility ranking | Risky choice prediction · Risky choice task | utility ranking agreement: 92.277% ± 0.165% | Helped |
Surprising findings
- On Car-Alt, the CRCS model performs worse than LCL, attributed to insufficient choice sets for training the surrogate.
- Without regularization, rankings implied by LCL, CRCS and LC-CRCS were inconsistent with collected human rankings (tau of -0.36, -0.17, -0.04).
Limitations
- Surrogates require sufficiently many choice sets or a well-specified task to generate training option sets; poor performance on Car-Alt.
- CRCS and LC-CRCS only support fixed-size choice sets.
- Non-convex likelihood requires repeated inference from multiple starting points.
Experimental setup
Datasets: Hotels District-Smart Car-Alt Dumbalska Journal of Medicinal Chemistry synthetic routes (47,055)
Benchmarks: Hotels District-Smart Car-Alt Dumbalska Crash structure design Water drainage network design Retrosynthesis planning
Metrics: negative log-likelihood Kendall's tau expected likelihood recommendation regret inference error maximum true utility
Baselines: Bradley-Terry Bower & Balzano Linear Context Logit (LCL) Aizynthfinder
Builds on: Howes et al. computationally rational choice model Linear Context Logit (LCL) Bradley-Terry Bower & Balzano model
Closest papers by meaning
- Preference Learning with Response Time: Robust Losses and Guarantees (NeurIPS 2025 · similarity 0.90)
- Enhancing Preference-based Linear Bandits via Human Response Time (NeurIPS 2024 · similarity 0.89)
- Capturing Individual Human Preferences with Reward Features (NeurIPS 2025 · similarity 0.89)
- ResponseRank: Data-Efficient Reward Modeling through Preference Strength Learning (NeurIPS 2025 · similarity 0.88)
- Distributional Preference Learning: Understanding and Accounting for Hidden Context in RLHF (ICLR 2024 · similarity 0.87)
- DeepHalo: A Neural Choice Model with Controllable Context Effects (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.