Research map / Reinforcement learning / RL from feedback
Learning Correlated Reward Models: Statistical Barriers and Opportunities
Yeshwanth Cherapanamjeri, Constantinos Costis Daskalakis, Gabriele Farina, Sobhan Mohammadpour
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train the probit with best-of-three instead of pairwise observations | correlated probit RUM · Synthetic (mu=0, Sigma=bin) | median accuracy: 0.50 -> 0.79 | Improved |
| train the probit with best-of-three instead of pairwise observations | correlated probit RUM · Synthetic (mu=0, Sigma=r) | median accuracy: 0.51 -> 0.67 | Improved |
| train the probit with best-of-three instead of pairwise observations | correlated probit RUM · Synthetic (mu=r, Sigma=I) | median accuracy: 0.66 -> 0.70 | Improved |
| train the probit with best-of-three instead of pairwise observations | correlated probit RUM · Synthetic (mu=r, Sigma=bin) | median accuracy: 0.95 -> 0.95 | No clear effect |
| train the probit with best-of-three instead of pairwise observations | correlated probit RUM · Sushi-A | median accuracy: 0.65 -> 0.68 | Improved |
| train the probit with best-of-three instead of pairwise observations | correlated probit RUM · Netflix 150k (onehot) | median accuracy: 0.56 -> 0.61 | Improved |
| train the probit with best-of-three instead of pairwise observations | correlated probit RUM · MovieLens 50k (LLM features) | median accuracy: 0.55 -> 0.59 | Improved |
| use correlated probit (best-of-three) instead of logit | reward/choice model · Sushi-A | median accuracy: 0.66 -> 0.68 | Improved |
| use correlated probit (best-of-three) instead of logit | reward/choice model · Netflix 150k (onehot) | median accuracy: 0.61 -> 0.61 | No clear effect |
| use correlated probit (best-of-three) instead of logit | reward/choice model · Synthetic (mu=0, Sigma=bin) | median accuracy: 0.50 -> 0.79 | Improved |
| use LLM embedding features instead of onehot features | probit (best-of-three) · Netflix 150k | median accuracy: 0.61 -> 0.61 | No clear effect |
| use LLM embedding features instead of onehot features | probit (best-of-three) · MovieLens 50k | median accuracy: 0.59 -> 0.59 | No clear effect |
Closest papers by meaning
- Capturing Individual Human Preferences with Reward Features (NeurIPS 2025 · similarity 0.90)
- Data with High and Consistent Preference Difference Are Better for Reward Model (AAAI 2025 · similarity 0.89)
- Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learning (NeurIPS 2024 · similarity 0.89)
- Preference Learning with Response Time: Robust Losses and Guarantees (NeurIPS 2025 · similarity 0.88)
- Preference Learning of Latent Decision Utilities with a Human-like Model of Preferential Choice (NeurIPS 2024 · similarity 0.88)
- Towards Theoretical Understanding of Sequential Decision Making with Preference Feedback (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
