Research map / Probabilistic and scientific modeling / Variational and generative sampling
MAVRL: Learning Reward Functions from Multiple Feedback Types with Amortized Variational Inference
Raphaël Baur, Yannick Metz, Maria Gkoulta, Mennatallah El-Assady, Giorgia Ramponi, Thomas Kleine Buening
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add stop feedback to demonstrations (ablation) | MAVRL · grid_sparse | normalized return: 95.0 -> 100.0 | Improved |
| combine feedback types jointly instead of post-hoc averaging | MAVRL · grid_cliff | normalized return: 54.9 -> 100.0 | Improved |
| combine feedback types jointly instead of post-hoc averaging | MAVRL · LunarLander-v3 | normalized return: 59.9 -> 93.5 | Improved |
| use amortized variational inference instead of MCMC | MAVRL · grid environments | wall time: 156.0 s -> 4.9 s on grid_cliff | Improved |
| use amortized variational inference instead of MCMC | MAVRL · grid_trap | normalized return: 93.0 -> 95.3 | No clear effect |
| combine all four feedback types instead of single modality | MAVRL · LunarLander-v3 | normalized return: 115.7 -> 93.5 | Hurt |
| combine all four feedback types instead of single modality | MAVRL · grid_cliff | EPIC distance: lowest EPIC 0.481 | Improved |
| add multi-type feedback under preference misspecification | MAVRL · grid_cliff | retained performance ratio: 0.19x -> 0.90x | Improved |
| add multi-type feedback when demonstrations are corrupted | MAVRL · grid_cliff and grid_sparse | retained performance ratio: 0.38-0.50 ratios | Hurt |
| use equal total feedback budget across modalities (ablation) | MAVRL · tabular grids | normalized return: single informative modality at full budget often beats multi-modal allocation | Mixed |
| use equal total feedback budget across modalities (ablation) | MAVRL · continuous-control | normalized return | Improved |
| combine feedback types instead of imitation under dynamics perturbation | MAVRL · LunarLander-v3 gravity and wind | normalized return | Improved |
Closest papers by meaning
- Reward Learning from Multiple Feedback Types (ICLR 2025 · similarity 0.90)
- Information-Theoretic Reward Decomposition for Generalizable RLHF (NeurIPS 2025 · similarity 0.88)
- Bayesian Preference Learning for Test-Time Steerable Reward Models (NeurIPS 2026 · similarity 0.88)
- A Unified Linear Programming Framework for Offline Reward Learning from Human Demonstrations and Feedback (ICML 2024 · similarity 0.87)
- Reward Model Learning vs. Direct Policy Optimization: A Comparative Analysis of Learning from Human Preferences (ICML 2024 · similarity 0.86)
- Active Reward Modeling: Adaptive Preference Labeling for Large Language Model Alignment (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
