Research map / Large language models / Reasoning and chain of thought

Optimizing Diversity and Quality through Base-Aligned Model Collaboration

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Routing, pruning and distillation for reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes BACo, an inference-time token-level collaboration between a base LLM and its aligned counterpart with uncertainty- and content-based routers, improving the diversity-quality trade-off in a single decoding pass.

Problem

Alignment improves LLM output quality but collapses output diversity across repeated samples, and existing diversity-promoting methods either degrade quality or require costly decoding or post-training.

Approach

Dynamically route each word-level decision between the base model (for diversity) and the aligned model (for quality) using logit-based (max probability, entropy) and content-based (punctuation, function-word) routing signals, with a threshold giving controllability along the diversity-quality frontier.

Key results

• With its best router, BACo achieves a 21.3% joint diversity-quality improvement over the strongest inference-time baselines, with Coverage improving 0.142 and Dominance 32.7% overall.
• On NoveltyBench, BACo's Coverage improves by 0.274 with Dominance 39.9%; semantic Dominance reaches 40.5%.
• Human raters rate BACo's quality higher than the aligned model on NoveltyBench (4.04 vs 2.83) despite near-identical automatic quality scores.

Surprising findings

• Random routing achieves competitive aggregate scores, but this comes almost entirely from lexical metrics and collapses on semantic diversity.
• The LLM-based judge router underperforms simple heuristic routers while costing more compute.

Limitations

• Two forward passes per decoding step adds FLOPs and latency overhead; prototype not engineered for speed.
• Early stopping failure mode when router is tuned toward diversity.
• Evaluation limited to open-ended tasks; agent and code generation not studied; learned routers left for future work.

Experimental setup

Datasets: NoveltyBench WildChat Narrative-Discourse IFEval GSM8K MT-Bench Artificial Hivemind

Benchmarks: NoveltyBench WildChat Narrative-Discourse IFEval GSM8K MT-Bench Artificial Hivemind

Metrics: Coverage Dominance Distinct-n EAD-n Self-BLEU Self-ROUGE-L embedding cosine dissimilarity Vendi Score NLI diversity semantic entropy

Baselines: Diverse Beam Search In-context resampling Paraphrase prompting Response ensemble Logits ensemble Nudging Min-p Top-p Multi-agent debate Mix-of-agents Back translation

Builds on: nudging superficial alignment hypothesis diverse beam search NoveltyBench

Closest papers by meaning

• SkipGPT: Each Token is One of a Kind (ICML 2025 · similarity 0.86)
• A Probabilistic Perspective on Unlearning and Alignment for Large Language Models (ICLR 2025 · similarity 0.86)
• Collab: Controlled Decoding using Mixture of Agents for LLM Alignment (ICLR 2025 · similarity 0.86)
• Towards Scalable Data Diversification for Language Model Pretraining via Leverage Score Sampling (NeurIPS 2026 · similarity 0.86)
• Aligner: Efficient Alignment by Learning to Correct (NeurIPS 2024 · similarity 0.86)
• Diversity as a Reward: Fine-Tuning LLMs on a Mixture of Domain-Undetermined Data (NeurIPS 2025 · similarity 0.86)