Research map / Large language models / Alignment and preferences
Robust Reward Modeling via Causal Rubrics
Pragya Srivastava, Harman Singh, Rahul Madhavan, Gandharv Patil, Sravanti Addepalli, Arun Suggala, Rengarajan Aravamudhan, Soumya Sharma, Anirban Laha, Aravindan Raghuveer, Karthikeyan Shanmugam, Doina Precup
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train with causal and neutral augmentations (Crome) instead of RRM | reward model training · RewardBench PairPM (Gemma-2-9B) | average accuracy: 82.54 -> 87.84 | Improved |
| train with causal and neutral augmentations (Crome) instead of RRM | reward model training · RewardBench PairPM (Gemma-2-9B) Safety | accuracy: 74.70 -> 87.14 | Improved |
| train with Crome instead of RRM | reward model training · RewardBench PairPM (Gemma-2-2B) | average accuracy: 66.23 -> 70.69 | Improved |
| train with Crome instead of RRM | reward model training · RewardBench PairPM Chat (Qwen2.5-7B) | accuracy: 97.21 -> 96.37 | Hurt |
| train with Crome instead of RRM | reward model training · RewardBench BT Chat-Hard (Gemma-2-9B) | accuracy: 69.15 -> 65.83 | Hurt |
| train with Crome instead of RRM | reward model training · reWordBench PairPM (Gemma-2-9B) | average accuracy: 64.68 -> 72.71 | Improved |
| train with Crome instead of RRM | reward model training · reWordBench BT (Gemma-2-9B) | average accuracy: 65.69 -> 69.81 | Improved |
| train with Crome instead of RRM | reward model training · RewardBench to reWordBench accuracy drop (PairPM) | percentage drop: 21.54% -> 19.78% | Improved |
| use Crome reward model for Best-of-N selection instead of RM | Best-of-N selection · WildGuardTest (N=32) | attack success rate: 42.11 -> 39.39 | Improved |
| use Crome reward model for Best-of-N instead of RM | Best-of-N selection · RewardBench BoN win rate (N=32) | win rate vs RM: 44.79 vs 21.88 | Improved |
| use Gemma-3-27B instead of Gemini 2.0 Flash as oracle LLM for augmentations | Crome oracle choice · RewardBench PairPM (Gemma-2-9B) | average accuracy: 82.53 -> 85.15 | Improved |
| add causal upgrade/degrade pairs plus irrelevant-query neutrals (IQN) to causal-only augmentation (ablation) | Crome · RewardBench Chat-Hard | accuracy | Improved |
Closest papers by meaning
- RRM: Robust Reward Model Training Mitigates Reward Hacking (ICLR 2025 · similarity 0.92)
- Interpreting Language Reward Models via Contrastive Explanations (ICLR 2025 · similarity 0.89)
- RMB: Comprehensively benchmarking reward models in LLM alignment (ICLR 2025 · similarity 0.89)
- Inference-Time Reward Hacking in Large Language Models (NeurIPS 2025 · similarity 0.89)
- Rethinking Reward Models for Multi-Domain Test-Time Scaling (NeurIPS 2026 · similarity 0.89)
- Linking Process to Outcome: Conditional Reward Modeling for LLM Reasoning (ICLR 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
