Research map / Large language models / Language, safety and interpretability
Prismatic Synthesis: Gradient-based Data Diversification Boosts Generalization in LLM Reasoning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes G-Vendi, a diversity metric based on the entropy of off-the-shelf model gradients that predicts LLM OOD generalization, and Prismatic Synthesis, a gradient-cluster-guided synthetic data generation loop that yields PrismMath-7B and PrismNLI.
Problem
Existing data diversity metrics (n-gram, embedding, perplexity, skill-set entropy) correlate weakly with how models generalize to unseen reasoning and NLI benchmarks, and naive synthetic data scaling saturates early.
Approach
Measure diversity as the exponentiated entropy (Vendi score) of normalized, randomly projected loss gradients from a small off-the-shelf proxy model, and use it to iteratively generate synthetic data targeting sparse clusters in gradient space.
Key results
- G-Vendi achieves Spearman's rho of 0.899 with math OOD performance and 0.893 with NLI OOD performance, outperforming Embedding Vendi (0.754 / 0.841).
- PrismMath-7B (distilled from R1-32B) reaches 57.08% on AIME24 and average 75.25, beating R1-Distill-Qwen-7B (72.80) on 6 of 7 benchmarks.
- PrismNLI (515K) outperforms the best prior mixture {WANLI+MNLI+SNLI} by about 8% average OOD accuracy (74.28 vs 66.18).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Prismatic Synthesis to diversify training data in gradient space | Qwen2.5-Math-7B-Instruct · Math reasoning benchmarks (7-benchmark average) | pass@1: 72.80 -> 75.25 | Helped |
| Use Prismatic Synthesis to diversify training data in gradient space | DeBERTa-v3-large · NLI OOD benchmarks (8-benchmark average) | accuracy: 66.18 -> 74.28 | Helped |
| Measure diversity with G-Vendi instead of Skill Set Entropy | Math reasoning data subsets · Math OOD performance (100K scale) | Spearman's ρ: 0.812 -> 0.899 | Helped |
| Measure diversity with G-Vendi instead of Embedding Vendi | NLI data subsets · NLI OOD performance (100K scale) | Spearman's ρ: 0.841 -> 0.893 | Helped |
| Replace the instruction-tuned gradient proxy with a base model (ablation) | G-Vendi on math reasoning data · Math OOD performance | rank correlation | Hurt |
| Replace the Qwen2.5-0.5B-Instruct gradient proxy with Llama-3.2-1B-Instruct (ablation) | G-Vendi on math reasoning data · Math OOD performance | rank correlation | Helped |
| Generate persona-diversified data instead of reasoning-pattern-diverse data | Math training data · Math OOD performance | OOD Perf.: 54.77 (3.49) -> 38.15 (8.02) | Hurt |
| Scale vanilla few-shot or persona-guided synthetic data beyond 50K–100K without gradient diversification (ablation) | Qwen2.5-7B-Math · Math reasoning benchmarks | average test accuracy | No clear effect |
Surprising findings
- Persona-guided generation beats vanilla few-shot only at low data scale and then plateaus, while the gradient-diversified pipeline keeps improving beyond 100K.
- Embedding-based diversity assigned higher scores to the persona-diversified dataset even though it generalized worse, whereas G-Vendi ranked the reasoning-diverse dataset higher.
Limitations
- G-Vendi is only meaningful when data quality is controlled; it cannot compare datasets from entirely different generation pipelines.
- OOD performance is an average over multiple benchmarks, which may obscure effects in new domains without benchmarks.
- Results rely on synthetic data from large teacher LLMs and majority-vote filtering rather than ground-truth verification.
Experimental setup
Datasets: WANLI GSM8k MATH OpenR1-Math PersonaHub MNLI SNLI FEVER
Benchmarks: AIME24 AIME25 AMC23 MATH500 MATH^2 OlympiadBench GSM8K Platinum HANS WNLI ANLI Diagnostics BigBench NLI
Metrics: Spearman's rho R^2 pass@1 relative accuracy
Baselines: Embedding Vendi Embedding Dissimilarity 2-gram Entropy Average Perplexity Skill Set Entropy R1-Distill-Qwen-7B OpenThinker2-7B OpenThinker-7B OpenR1-Qwen-7B WANLI MNLI MNLI+FEVER
Builds on: Vendi score influence functions (Pruthi et al.) persona-guided synthetic data generation OpenR1-Math WANLI
Closest papers by meaning
- Diversity-Aware Policy Optimization for Large Language Model Reasoning (NeurIPS 2025 · similarity 0.91)
- What Do Learning Dynamics Reveal About Generalization in LLM Mathematical Reasoning? (ICML 2025 · similarity 0.90)
- Exploring the Limits of Sub-Billion Language Model Reasoners with Open Training Recipes (ICLR 2026 · similarity 0.90)
- Turning Dust into Gold: Distilling Complex Reasoning Capabilities from LLMs by Leveraging Negative Data (AAAI 2024 · similarity 0.90)
- It's All Training: A Fully Synthetic Single-Stage Recipe for LLMs (NeurIPS 2026 · similarity 0.90)
- Flow of Reasoning: Training LLMs for Divergent Reasoning with Minimal Examples (ICML 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.