Research map / Large language models / Reasoning and chain of thought
DISC: Dynamic Decomposition Improves LLM Inference Scaling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DISC, a recursive inference-time method that dynamically partitions LLM solutions into steps of adaptive size (z-score-based acceptance), improving pass@k on APPS, MATH500 and LiveCodeBench over token-, line-, and single-step decomposition.
Problem
Inference scaling methods decompose problems into fixed or hand-designed steps (tokens, sentences, whole solutions), which wastes compute on easy steps and undersamples difficult ones.
Approach
Adaptively partition solution traces during inference: sample completions from a base prefix, estimate the z-score of the sample maximum, and accept an extended candidate prefix only if its z-score is lower; otherwise contract the step size and resample. This needs only an outcome reward model and plugs into greedy, beam, or MCTS search.
Key results
- On APPS pass@10 error decreases from 0.50 to 0.475 (5.0% relative reduction vs best baseline), MATH500 0.15 to 0.14 (6.7%), LiveCodeBench 0.57 to 0.51 (10.5%).
- For Llama pass@10 rises from 0.01 to 0.04 (300% relative); Qwen from 0.095 to 0.17 (79% relative).
- With R1 on APPS, DISC improves accuracy by over 85% relative to base R1 using 10 samples.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace the best static decomposition baseline with dynamic decomposition | gpt-4o-mini · APPS | Pass@10 error proportion: 0.50 -> 0.475 | Helped |
| Replace the best static decomposition baseline with dynamic decomposition | gpt-4o-mini · MATH500 | Pass@10 error proportion: 0.15 -> 0.14 | Helped |
| Replace the best static decomposition baseline with dynamic decomposition | gpt-4o-mini · LiveCodeBench | Pass@10 error proportion: 0.57 -> 0.51 | Helped |
| Use lower sampling temperature (ablation) | DISC · APPS | Pass@token | Helped |
| Use z-score-based acceptance instead of alternative acceptance criteria (ablation) | DISC · APPS competition level | token-level performance | Helped |
| Use higher z-score for candidate acceptance instead of lower z-score (ablation) | DISC · APPS competition level | token-level performance | Hurt |
| Set initial partition fraction within 0.15–0.25 (ablation) | DISC · APPS | performance | Helped |
| Set initial partition fraction above 0.3 (ablation) | DISC · APPS | accuracy | Hurt |
Surprising findings
- DISC performs better at lower sampling temperatures, in contrast to BoN which peaks at 0.6-0.8.
- Z-score acceptance performs worse than random selection when inverted (negZ), showing acceptance direction matters a lot.
Limitations
- Assumes access to a reasonably accurate reward model; code tasks need ground-truth or self-generated tests.
- Theoretical optimality result relies on a strong, instance-dependent policy assumption.
- Evaluated on subsets (200 APPS problems, 500 MATH500, 108 LiveCodeBench) due to compute constraints.
Experimental setup
Datasets: APPS MATH LiveCodeBench
Benchmarks: APPS MATH500 LiveCodeBench
Metrics: pass@k pass@token error proportion
Baselines: TokenSplit LineSplit BoN
Builds on: best-of-n sampling Monte Carlo Tree Search beam search
Closest papers by meaning
- Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for LLM Problem-Solving (ICLR 2025 · similarity 0.91)
- The Silent Hyperparameter: Quantifying the Impact of Inference Backends on LLM Reproducibility (NeurIPS 2026 · similarity 0.91)
- Bag of Tricks for Inference-time Computation of LLM Reasoning (NeurIPS 2025 · similarity 0.90)
- Efficient Test-Time Scaling via Hierarchical Search and Self-Verification for Discrete Diffusion Language Models (ICML 2026 · similarity 0.90)
- xLSTM 7B: A Recurrent LLM for Fast and Efficient Inference (ICML 2025 · similarity 0.89)
- Wider or Deeper? Scaling LLM Inference-Time Compute with Adaptive Branching Tree Search (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.