Research map / Large language models / Instruction and fine-tuning
Corrector Sampling in Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Resample-Previous-Tokens (RPT), a sampling method that iteratively revisits and resamples a window of previously generated tokens using a previous-token-prediction conditional learned via lightweight fine-tuning of an 8B autoregressive model.
Problem
Autoregressive language models cannot revise tokens once sampled, so errors accumulate during left-to-right next-token-prediction (NTP) sampling.
Approach
Train the model with permuted inputs so it also learns previous-token-prediction conditionals p(x_i | x_<i, x_{i+1}), then at inference iteratively resample a window of previously generated tokens conditioned on future tokens; theory bounds the sampling error via a Markov chain perturbation argument.
Key results
- RPT sampling (k=1, w=2) with fine-tuned 8B model improves GSM8K from 35.2 (AR-F) to 37.5 and MBPP from 39.0 to 40.6, with about 10% relative gains on reasoning and coding tasks.
- Empirical TV distance on validation data drops from 0.34 to 0.30 on dclm after one RPT iteration.
- RPT improves the ground-truth token probability over NTP in 64.5% of DCLM validation cases.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply RPT sampling with k=1 after 100B-token fine-tuning | 8B autoregressive model · HumanEval+ | score: 25.6 -> 27.4 | Helped |
| Apply RPT sampling with k=1 after 100B-token fine-tuning | 8B autoregressive model · MBPP | score: 39.0 -> 40.6 | Helped |
| Apply RPT sampling with k=1 after 100B-token fine-tuning | 8B autoregressive model · GSM8K | score: 35.2 -> 37.5 | Helped |
| Apply RPT sampling with k=1 after 100B-token fine-tuning | 8B autoregressive model · MultiPL-E C++ | score: 28.5 -> 31.0 | Helped |
| Apply RPT sampling with k=1 after 100B-token fine-tuning | 8B autoregressive model · MultiPL-E C# | score: 17.7 -> 22.2 | Helped |
| Apply RPT sampling with k=1 after 100B-token fine-tuning | 8B autoregressive model · MultiPL-E PHP | score: 28.5 -> 25.5 | Hurt |
| Apply RPT sampling with k=1 after 100B-token fine-tuning | 8B autoregressive model · MultiPL-E Bash | score: 6.9 -> 9.5 | Helped |
| Apply RPT sampling with k=1 after 100B-token fine-tuning | 8B autoregressive model · MultiPL-E Java | score: 37.9 -> 41.1 | Helped |
| Apply RPT sampling with k=1 after 100B-token fine-tuning | 8B autoregressive model · MultiPL-E TypeScript | score: 35.8 -> 38.4 | Helped |
| Perform one RPT iteration instead of k=0 NTP | RPT-trained 8B model · DCLM validation | total variation distance: .34 -> .30 | Helped |
| Perform one RPT iteration instead of k=0 NTP | RPT-trained 8B model · GitHub validation | total variation distance: .18 -> .15 | Helped |
| Perform one RPT iteration instead of k=0 NTP | RPT-trained 8B model · Wiki validation | total variation distance: .26 -> .23 | Helped |
Surprising findings
- Even NTP sampling (k=0) from the fine-tuned model outperforms the fully trained baseline AR-F on most benchmarks.
Limitations
- RPT sampling is more computationally expensive than NTP sampling.
- Evaluation relies on oracle selection over temperatures, which is a favorable comparison protocol.
- Only window sizes 2 and 3 explored, with no clear practical benefit from w=3.
Experimental setup
Datasets: proprietary 1T-token language corpus DCLM GitHub Wikipedia arXiv Stack Exchange
Benchmarks: HumanEval+ MBPP GSM8K MultiPL-E
Metrics: pass@1 total variation distance
Baselines: NTP sampling AR-F AR-C Beam Search
Builds on: relative positional encoding
Closest papers by meaning
- Beyond Autoregression: Fast LLMs via Self-Distillation Through Time (ICLR 2025 · similarity 0.86)
- Fast and Fluent Diffusion Language Models via Convolutional Decoding and Rejective Fine-tuning (NeurIPS 2025 · similarity 0.86)
- Hierarchical Autoregressive Transformers: Combining Byte- and Word-Level Processing for Robust, Adaptable Language Models (ICLR 2025 · similarity 0.86)
- Breaking AR’s Sampling Bottleneck: Provable Acceleration via Diffusion Language Models (NeurIPS 2025 · similarity 0.86)
- Closing the Curious Case of Neural Text Degeneration (ICLR 2024 · similarity 0.85)
- Not All Tokens Are What You Need for Pretraining (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.