Research map / Large language models / Reasoning and chain of thought

SLED: Self Logits Evolution Decoding for Improving Factuality in Large Language Models

NeurIPS 2024 · Large language models › Reasoning and chain of thought · approach: Theory of autoregressive reasoning · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SLED, a training-free decoding method that contrasts final-layer logits with early-layer logits to estimate a latent factual distribution and applies a single approximate gradient step to refine the output logits, improving factuality on TruthfulQA and FACTOR.

Problem

LLMs often produce factually incorrect outputs even when they latently encode the correct facts; the goal is to use this latent knowledge at inference time to improve factuality without external knowledge bases or fine-tuning.

Approach

Treats the difference between early-layer and final-layer logits as an approximate gradient direction of a KL divergence to the real-world distribution, estimates a latent target distribution by cosine-aligning this direction with one-hot gradients across all early layers (soft, weighted ensemble), and performs one gradient-descent step on the final logits restricted to the top-k tokens.

Key results

• On Gemma-3-1B-PT, SLED raises FACTOR accuracy from 47.83 (greedy) and 12.63 (DoLa) to 63.29.
• On Gemma-3-27B-IT, SLED reaches 73.75 FACTOR and 47.47 TruthfulQA MC1 versus 66.00 and 41.14 for greedy decoding.
• On Mixtral-8x7B, SLED improves FACTOR from 71.41 to 74.92 and MC2 from 49.98 to 57.26 over greedy decoding.

Measured effects reported in the paper

Surprising findings

• DoLa often underperforms vanilla greedy decoding on FACTOR for Gemma-3 models, dropping FACTOR accuracy by large margins (e.g., 47.83 to 12.63 on Gemma-3-1B-PT).
• A larger DoLa candidate layer set (0-32) performs worse than 0-16, which the authors take as evidence of fragility in layer selection.

Limitations

• Relies on the assumption that early-layer logit differences approximate gradient directions, supported only by empirical cosine-similarity analysis.
• Hyperparameters (evolution rate, scale k) are tuned on a ~5% validation sample of each benchmark.
• Evaluated only on multiple-choice factuality benchmarks; open-ended generation and CoT claims are not clearly backed by reported results in the provided text.

Experimental setup

Datasets: TruthfulQA FACTOR (Wiki)

Benchmarks: TruthfulQA FACTOR (Wiki)

Metrics: MC1 MC2 MC3 FACTOR accuracy

Baselines: DoLa greedy decoding

Builds on: DoLa contrastive decoding

Closest papers by meaning

• Sample Smart, Not Hard: Correctness-First Decoding for Better Reasoning in LLMs (ICLR 2026 · similarity 0.89)
• DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models (ICLR 2024 · similarity 0.89)
• SelfJudge: Faster Speculative Decoding via Self-Supervised Judge Verification (ICML 2026 · similarity 0.88)
• Small Language Model Can Self-Correct (AAAI 2024 · similarity 0.88)
• Latent Self-Consistency for Reliable Majority-Set Selection in Short- and Long-Answer Reasoning (AAAI 2026 · similarity 0.87)
• Scalable Best-of-N Selection for Large Language Models via Self-Certainty (NeurIPS 2025 · similarity 0.87)