Research map / Adversarial robustness and security / Attacks on text and graphs
Reward Under Attack: Analyzing the Robustness and Hackability of Process Reward Models
Rishabh Tiwari, Aditya Tomar, Udbhav Bamba, Monishwaran Maheswaran, Heng Yang, Michael W. Mahoney, Kurt Keutzer, Amir Gholami
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| append 100 optimized adversarial tokens | Skywork-o1-Open-PRM-1.5B · AIME25 transfer | mean PRM reward: 0.305 -> 0.924 | Improved |
| append 50 optimized adversarial tokens | Skywork-o1-Open-PRM-1.5B · AIME25 transfer | mean PRM reward: +0.224 | Improved |
| append 50 optimized adversarial tokens | Skywork-o1-Open-PRM-7B · AIME25 transfer | mean PRM reward: +0.070 | Improved |
| append 100 optimized adversarial tokens | Qwen2.5-Math-PRM-7B · AIME24 training reward | best training reward: 0.658 -> 0.437 | Hurt |
| append 100 optimized adversarial tokens | Qwen2.5-Math-PRM-7B · AIME25 transfer | reward change: -0.042 | Hurt |
| increase adversarial tokens from 1 to 100 | Skywork-o1-Open-PRM-1.5B · AIME25 transfer | reward change: +0.030 -> +0.619 | Improved |
| use adversarial rather than random tokens (100 tokens) | Skywork-o1-Open-PRM-1.5B · AIME24 reward landscape | basin volume: 1.495 vs 0.689 | Improved |
| use adversarial rather than random tokens (100 tokens) | Qwen2.5-Math-PRM-7B · AIME24 reward landscape | basin volume: 1.570 vs 0.421 | Improved |
| train policy with GRPO on PRM reward | Qwen2.5-1.5B-Instruct · AIME25 | mean PRM reward (Skywork-1.5B): 0.246 -> 0.641 | Improved |
| GRPO training on PRM reward | Qwen2.5-1.5B-Instruct · AIME24 | ground-truth accuracy: near zero (peaking at 3-4%) | No clear effect |
| GRPO training on Qwen-7B PRM reward | Qwen2.5-1.5B-Instruct · AIME24 | ground-truth accuracy: accuracy drops to 0% | Hurt |
| rephrase GRPO trajectories (semantics-preserving) (ablation) | GRPO policy with Skywork-1.5B PRM · AIME25 | mean PRM reward: 0.641 -> 0.472 | Hurt |
Closest papers by meaning
- Adversarial Training for Process Reward Models (ICML 2026 · similarity 0.92)
- Stop Summation: Min-Form Credit Assignment Is All Process Reward Model Needs for Reasoning (NeurIPS 2025 · similarity 0.90)
- Linking Process to Outcome: Conditional Reward Modeling for LLM Reasoning (ICLR 2026 · similarity 0.89)
- Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoning (ICLR 2025 · similarity 0.89)
- From Mathematical Reasoning to Code: Generalization of Process Reward Models in Test-Time Scaling (AAAI 2026 · similarity 0.89)
- Causal Reward Adjustment: Mitigating Reward Hacking in External Reasoning via Backdoor Correction (AAAI 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
