Research map / Large language models / Reasoning and chain of thought
Semantic-guided Diverse Decoding for Large Language Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SemDiD, a training-free group beam-search decoding method that steers candidate sequences in embedding space (orthogonal directions, inter-group repulsion) under a quality constraint, improving semantic diversity for Best-of-N and RLHF.
Problem
Existing diverse decoding methods (temperature, nucleus, diverse beam search, n-gram penalties) produce lexical rather than semantic diversity, limiting Best-of-N, group-based RL, and data synthesis.
Approach
Runs k groups of beam search where each diverse group is guided by an orthogonal direction in a sentence-embedding space plus a dynamic inter-group repulsion term, with a position-debiased probability as quality lower bound enforced via an epsilon-constraint and combined by a harmonic gain.
Key results
- Best-of-N coverage with 25 samples on Qwen-2.5-3B: 82.4% ARC-Challenge, 85.6% BBH, 98.1% GSM8K, 86.1% Minerva Math.
- SemDiD improves Best-of-N coverage by 1.4-5.2% across tasks, per the abstract.
- RLHF with Qwen-2.5-7B: GSM8K accuracy 88.2% with GRPO and TLDR win rate 73.4% with GRPO at 60 rollouts; abstract claims 15% faster convergence and up to 2.1% higher accuracy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| select semantically clustered candidates instead of highest-probability candidates | Best-of-N candidate selection · GSM8K, BBH; top-25 from 100 samples | coverage: 92% to 95% | Helped |
| use SemDiD instead of other decoding methods | Best-of-N decoding · ARC-Challenge, 25 samples | coverage | Helped |
| use SemDiD instead of other decoding methods | Best-of-N decoding · GSM8K, 25 samples | coverage | Helped |
| use SemDiD for rollouts instead of other decoding methods | RLHF training algorithms · GSM8K | accuracy | Helped |
| apply position debiasing to raw probability (ablation) | SemDiD quality assessment · Probability debiasing ablation, 10 samples per question | Avg. Coverage: 67.85% -> 68.41% | Helped |
| apply position debiasing to raw probability (ablation) | SemDiD quality assessment · Probability debiasing ablation, 10 samples per question | Avg. Accuracy by LLM-as-Judge: 61.02% -> 61.90% | Helped |
| apply sentence debiasing to raw probability (ablation) | SemDiD quality assessment · Probability debiasing ablation, 10 samples per question | Avg. Coverage: 67.85% -> 67.95% | Helped |
| apply sentence debiasing to raw probability (ablation) | SemDiD quality assessment · Probability debiasing ablation, 10 samples per question | Avg. Accuracy by LLM-as-Judge: 61.02% -> 61.23% | Helped |
| combine position and sentence debiasing (ablation) | SemDiD quality assessment · Probability debiasing ablation, 10 samples per question | Avg. Coverage: 67.85% -> 68.64 % | Helped |
| combine position and sentence debiasing (ablation) | SemDiD quality assessment · Probability debiasing ablation, 10 samples per question | Avg. Accuracy by LLM-as-Judge: 61.02% -> 62.36% | Helped |
| add a saturation threshold to combined debiasing (ablation) | SemDiD quality assessment · Probability debiasing ablation, 10 samples per question | Avg. Coverage: 68.64 % -> 68.76 % | Helped |
| add a saturation threshold to combined debiasing (ablation) | SemDiD quality assessment · Probability debiasing ablation, 10 samples per question | Avg. Accuracy by LLM-as-Judge: 62.36% -> 62.47% | Helped |
Surprising findings
- Probability-based quality thresholding loses accuracy when log-probability threshold exceeds -1, since many correct answers are misclassified; probability is better used as a lower bound than as a maximization target.
- Embedding model size has minimal impact on semantic clustering diversity (0.5B vs 1.5B differ by 0.1% coverage at k=10).
Limitations
- Added 25-35% latency overhead relative to group beam search from embedding computation and lookahead.
- Probability-based quality assessment is an imperfect proxy, with accuracy degrading at high thresholds.
- Theoretical guarantees rest on loosely specified quantities (e.g., maximum quality dispersion, semantic variance) and are informal.
Experimental setup
Datasets: GSM8K TLDR ARC-Challenge BBH Minerva Math CoQA PubMedQA MMLU-Pro+ WMT16
Benchmarks: ARC-Challenge BBH GSM8K Minerva Math CoQA PubMedQA MMLU-Pro+ WMT16 En-De WMT16 De-En TLDR
Metrics: coverage accuracy win rate
Baselines: Temperature Sampling Arithmetic Sampling Diverse Beam Search Determinantal Beam Search Group Beam Search
Builds on: diverse beam search determinantal beam search group relative policy optimization RLOO iterative RLHF
Closest papers by meaning
- Large Language Models Explore by Latent Distilling (ICML 2026 · similarity 0.89)
- Entropy-informed Decoding: Adaptive Information-Driven Branching (ICML 2026 · similarity 0.87)
- XTC: Head-Aware Sampling by Excluding Top Choices (NeurIPS 2026 · similarity 0.87)
- SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding (ICML 2026 · similarity 0.87)
- Optimized Multi-Token Joint Decoding With Auxiliary Model for LLM Inference (ICLR 2025 · similarity 0.86)
- Diversity as a Reward: Fine-Tuning LLMs on a Mixture of Domain-Undetermined Data (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.