Research map / Large language models / Language, safety and interpretability
Synthesize, Partition, then Adapt: Eliciting Diverse Samples from Foundation Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Synthesize-Partition-Adapt (SPA), which partitions synthetic instruction data by influence-function or BM25 attribution scores and trains LoRA adaptations per partition so that sampling (even greedy) yields diverse yet accurate responses.
Problem
Foundation models produce low-diversity outputs under greedy decoding, and temperature sampling trades diversity for quality; users often want several diverse high-quality responses.
Approach
Partition synthetic instruction data using data attribution scores (influence function or lexical overlap) to test queries, train one LoRA adaptation per partition during post-training, and sample one response from a randomly chosen adaptation so diversity comes from the adaptations rather than sampling temperature.
Key results
- On HumanEval, influence-based SPA reaches diversity 0.85 and pass@5 69.05 versus 0.58 and 56.42 for the single-adaptation temperature-0.1 baseline, with pass@1 ~50.15 maintained.
- On MBPP, influence-based SPA reaches diversity 0.78, avg KL 0.020 and pass@5 73.68, with pass@1 60.46 comparable to the single baseline's 60.15.
- Influence-based partitioning consistently beats lexical overlap and random partitioning on diversity and KL across NLU tasks (BBH, GPQA, MMLU, WinoGrande).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use influence-based partitioning instead of a single adaptation | SPA on CodeLLaMA-7B-Python · HumanEval | diversity: 0.58 -> 0.85 | Helped |
| Use influence-based partitioning instead of random partitioning | SPA on CodeLLaMA-7B-Python · HumanEval | avg. KL: 0.008 -> 0.017 | Helped |
| Use influence-based partitioning instead of lexical-overlap partitioning | SPA on CodeLLaMA-7B-Python · HumanEval | pass@5: 66.74 -> 69.05 | Helped |
| Use influence-based partitioning instead of a single adaptation | SPA on CodeLLaMA-7B-Python · MBPP | diversity: 0.53 -> 0.78 | Helped |
| Use influence-based partitioning instead of random partitioning | SPA on CodeLLaMA-7B-Python · MBPP | avg. KL: 0.014 -> 0.020 | Helped |
| Use influence-based partitioning instead of lexical-overlap partitioning | SPA on CodeLLaMA-7B-Python · MBPP | pass@5: 71.17 -> 73.68 | Helped |
| Increase the number of adaptations from 8 to 12 (ablation) | SPA on HumanEval · HumanEval | diversity | No clear effect |
| Increase sampling temperature | SPA and baselines on HumanEval · HumanEval | pass@1: around 50.2% -> approximately 46.5% | Hurt |
| Increase sampling temperature | SPA and baselines on HumanEval · HumanEval | diversity | Helped |
Surprising findings
- Random partitioning into adaptations beats lexical overlap on GPQA diversity, suggesting partitioning benefit is task-dependent.
- Training more than 8 adaptations (up to 12) did not increase diversity score.
Limitations
- Influence function computation is expensive, requiring several extra backward passes over the dataset.
- Ranking heuristic produces imbalanced partitions; more advanced clustering is left for future work.
- Serving multiple LoRA adaptations adds real-time serving overhead.
Experimental setup
Datasets: OSS-Instruct Platypus hand-written test queries (12 per domain)
Benchmarks: HumanEval MBPP Big-Bench Hard GPQA MMLU WinoGrande
Metrics: pass@1 pass@5 sample diversity average KL divergence
Baselines: Single adaptation with temperature sampling Random partition multiple adaptations Lexical overlap (BM25) partitioning
Builds on: LoRA influence functions BM25 OSS-Instruct Platypus
Closest papers by meaning
- Diversity as a Reward: Fine-Tuning LLMs on a Mixture of Domain-Undetermined Data (NeurIPS 2025 · similarity 0.85)
- Boosting Multi-Domain Fine-Tuning of Large Language Models through Evolving Interactions between Samples (ICML 2025 · similarity 0.85)
- Towards Scalable Data Diversification for Language Model Pretraining via Leverage Score Sampling (NeurIPS 2026 · similarity 0.85)
- SimpleStrat: Diversifying Language Model Generation with Stratification (NeurIPS 2025 · similarity 0.85)
- Free Lunch for Pass@$k$? Low Cost Diverse Sampling for Diffusion Language Models (NeurIPS 2026 · similarity 0.84)
- Prismatic Synthesis: Gradient-based Data Diversification Boosts Generalization in LLM Reasoning (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.