Research map / Reinforcement learning / RL from feedback
Reward Shaping for (Inference-Time) Alignment: A Stackelberg Game Perspective
Haichuan Wang, Tao Lin, Lingkai Kong, Ce Li, Hezi Jiang, Milind Tambe
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add SRS threshold reward shaping | ARGS · Eval-1 (HH-RLHF, Qwen) | average reward: 3.23 -> 3.33 | Improved |
| add SRS threshold reward shaping | CD · Eval-2 (SHP, Qwen) | average reward: 3.10 -> 3.37 | Improved |
| add SRS threshold reward shaping | ARGS · Eval-3 (HH-RLHF, Llama) | average reward: 1.87 -> 2.04 | Improved |
| add SRS threshold reward shaping | ARGS · Eval-4 (SHP, Llama) | average reward: 2.97 -> 3.29 | Improved |
| replace unbounded Meanstd normalization with SRS | CD · Eval-2 (SHP, Qwen) | average reward: 2.65 -> 3.37 | Improved |
| replace Minmax normalization with SRS | ARGS · Eval-2 (SHP, Qwen) | average reward: 3.14 -> 3.40 | Improved |
| replace Minmax normalization with SRS | ARGS · Eval-4 (SHP, Llama) | average reward: 3.20 -> 3.29 | Improved |
| use SRS threshold instead of reward capping (ablation) | CD · Eval-2 (SHP, Qwen) | average reward: 2.96 -> 3.37 | Improved |
| use SRS threshold instead of reward capping (ablation) | ARGS · Eval-2 (SHP, Qwen) | average reward: 3.29 -> 3.40 | Improved |
| use SRS threshold instead of mean threshold (ablation) | ARGS · Eval-2 (SHP, Qwen) | diversity: 0.74 -> 0.81 | Improved |
| increase Monte Carlo samples M from 10 to 20 (ablation) | SRS-ARGS · Eval-1 (HH-RLHF, Qwen) | average reward: 3.33 -> 3.40 | Improved |
| increase Monte Carlo samples M from 10 to 20 (ablation) | ARGS · Eval-1 (HH-RLHF, Qwen) | average reward: 3.23 -> 3.20 | No clear effect |
Closest papers by meaning
- Direct Alignment with Heterogeneous Preferences (NeurIPS 2025 · similarity 0.91)
- Inference-Time Reward Hacking in Large Language Models (NeurIPS 2025 · similarity 0.90)
- Reward-Augmented Data Enhances Direct Preference Alignment of LLMs (ICML 2025 · similarity 0.90)
- Energy-Based Preference Model Offers Better Offline Alignment than the Bradley-Terry Preference Model (ICML 2025 · similarity 0.90)
- Scaling Laws for Reward Model Overoptimization in Direct Alignment Algorithms (NeurIPS 2024 · similarity 0.90)
- Data with High and Consistent Preference Difference Are Better for Reward Model (AAAI 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
