Research map / Large language models / Reasoning and chain of thought

Diversity Combining for Multi-Path LLM Reasoning

NeurIPS 2026 · Large language models › Reasoning and chain of thought · approach: Long CoT dynamics and efficiency · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Formalizes multi-path LLM reasoning (self-consistency) as diversity combining, derives a design-effect saturation ceiling 1/c from pairwise correctness correlation, and proposes an Adaptive-K rule that picks the number of paths from a four-path pilot.

Problem

Self-consistency gains plateau as the number of sampled reasoning paths K grows, and no existing method predicts the saturation point K* for a given model and task.

Approach

Treats reasoning paths as correlated diversity branches from wireless diversity combining; the vote-level effective sample size K/(1+(K-1)c) saturates at 1/c, with c measured from pilot correctness. Adds GLS analysis showing uniform weighting is optimal under exchangeability, and shows prompt-template diversity decorrelates paths, with the effect gated by answer-space openness.

Key results

• On GSM8K with Qwen2.5-7B, Keff^vote at K=32 reaches 98% of the 1/c ceiling (1.67 vs 1.71), with correctness correlation about 0.59.
• Prompt-template diversity reduces pairwise correlation in 55 of 57 valid cells, with mean relative decrease of 38% and QA cells decreasing about 67% versus 9-29% on math.
• Adaptive-K selected from a K=4 pilot retains 96-103% of MV@32 accuracy across GSM8K, HotpotQA, and BoolQ at 25-44% of the full K=32 cost.

Surprising findings

• Prompt-template diversity raises the saturation ceiling but does not improve majority-vote accuracy much; MV accuracy changes are small and mixed in sign.
• The independence binomial model badly overpredicts MV@32 (e.g. 100% vs 81.9% actual for Qwen-7B on GSM8K), while the beta-binomial tracks within about 1 pp.

Limitations

• Adaptive-K uses a single global correlation estimate and does not adapt to per-instance difficulty.
• GLS analysis needs hidden-state embeddings, which API-only deployments lack.
• Per-cell decorrelation estimates have wide confidence intervals; several math/code cells lack statistical power.

Experimental setup

Datasets: GSM8K HotpotQA TriviaQA DROP BoolQ ARC-C MMLU HellaSwag WinoGrande MBPP CruxEval MATH

Benchmarks: GSM8K HotpotQA TriviaQA DROP BoolQ ARC-C MMLU HellaSwag WinoGrande MBPP CruxEval MATH

Metrics: majority-vote accuracy correctness correlation design-effect effective sample size relative change in pairwise correlation retained accuracy

Baselines: self-consistency binomial independence model Adaptive-Consistency CISC

Builds on: self-consistency Kish design effect beta-binomial model Condorcet jury theorem generalized least squares

Closest papers by meaning

• A Theoretical Study on Bridging Internal Probability and Self-Consistency for LLM Reasoning (NeurIPS 2025 · similarity 0.87)
• HyPER: Bridging Exploration and Exploitation for Scalable LLM Reasoning with Hypothesis Path Expansion and Reduction (ICML 2026 · similarity 0.86)
• THE PATH OF LEAST RESISTANCE: GUIDING LLM REASONING TRAJECTORIES WITH PREFIX CONSENSUS (ICLR 2026 · similarity 0.86)
• R-Diverse: Mitigating Diversity Illusion in Self-Play LLM Training (ICML 2026 · similarity 0.86)
• Optimal Self-Consistency for Efficient Reasoning with Large Language Models (ICML 2026 · similarity 0.85)
• Flow of Reasoning: Training LLMs for Divergent Reasoning with Minimal Examples (ICML 2025 · similarity 0.85)