Research map / Large language models / Reasoning and chain of thought
ZIP-RC: Zero-overhead Inference-time Prediction of Reward and Cost for Adaptive and Interpretable Generation
Rohin Manvi, Joey Hong, Tim Seyde, Maxime Labonne, Mathias Lechner, Sergey Levine
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ZIP-RC, a zero-overhead inference-time method that lets LLMs predict their own success probability (reward) and remaining computation cost (cost) to guide adaptive generation decisions.
Problem
LLMs lack introspection: they cannot anticipate their own likelihood of success or the compute needed, which hampers decisions on effort allocation, multiple attempts, stopping, and signaling success or failure.
Approach
Equips the model to predict reward and cost at inference time with essentially no extra overhead, enabling interpretable, adaptive meta-cognitive control over generation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace majority voting with ZIP-RC sampling at matched cost | Qwen3-1.7B · AIME 2024 | accuracy: 53.1 -> 65.8 | Improved |
| replace majority voting with ZIP-RC sampling at matched cost | LFM2-350M · AIME 2024 | accuracy: 26.9 -> 38.8 | Improved |
| replace majority voting with ZIP-RC sampling at matched cost | LFM2-1.2B · Mixed-difficulty | accuracy: 83.8 -> 86.0 | Improved |
| replace majority voting with ZIP-RC sampling | Qwen3-1.7B · Mixed-difficulty | accuracy: 91.0 -> 92.2 | Improved |
| replace sampling-utility optimisation with reward-threshold pruning (ablation) | ZIP-RC · Mixed-difficulty (Qwen3-1.7B) | accuracy: 88.9 -> 92.2 | Improved |
| replace sampling-utility optimisation with reward-threshold pruning (ablation) | ZIP-RC · AIME 2024 (LFM2-1.2B) | accuracy: 57.5 -> 60.9 | Improved |
| ZIP-RC sampling versus majority voting on cost | ZIP-RC · Matched cost, LFM2-1.2B and LFM2-350M | generation cost: 1.60 -> 1.35; 1.70 -> 1.49 | Improved |
| ZIP-RC sampling versus majority voting on cost for Qwen3-1.7B | ZIP-RC · Matched cost | generation cost: 1.40 -> 1.43 | No clear effect |
| add length-based pruning to majority voting (ablation) | Majority voting · AIME 2024 (Qwen3-1.7B) | accuracy: 53.1 -> 25.1 | Hurt |
| keep full model trainable with KL regularisation instead of ZIP-RC-Lite head-only training (ablation) | ZIP-RC · Qwen3-1.7B end-of-generation prediction | accuracy: 0.71 -> 0.88 | Improved |
| ZIP-RC-Lite in compute-bound setting (ablation) | ZIP-RC-Lite · alpha=1.0 | compute efficiency | Hurt |
Closest papers by meaning
- Inference-Time Reward Hacking in Large Language Models (NeurIPS 2025 · similarity 0.89)
- On the Limits of Test-Time Compute: Sequential Reward Filtering for Better Inference (ICML 2026 · similarity 0.89)
- Think-RM: Enabling Long-Horizon Reasoning in Generative Reward Models (NeurIPS 2025 · similarity 0.88)
- Rethinking Reward Models for Multi-Domain Test-Time Scaling (NeurIPS 2026 · similarity 0.88)
- Optimizing Test-Time Compute via Meta Reinforcement Finetuning (ICML 2025 · similarity 0.88)
- Fixing the Broken Compass: Diagnosing and Improving Inference-Time Reward Modeling (ICLR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
