Research map / Large language models / Language, safety and interpretability
GEM: Geometric Entropy Mixing for Optimal LLM Data Curation
Yue Min, Ziyun Qiao, Ruining Chen, Yujun Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add vMF mixture with soft assignments instead of spherical K-Means (ablation) | GEM clustering · Clustering ablation | average accuracy: 40.6 -> 41.3 | Improved |
| add mixing-balance regularizer and entropy constraint to vanilla vMF (ablation) | GEM clustering · Clustering ablation | average accuracy: 41.3 -> 42.1 | Improved |
| use spherical cosine geometry instead of Euclidean K-Means (ablation) | clustering · Clustering ablation | average accuracy: 38.5 -> 40.6 | Improved |
| use GIS instead of random sampling for prototype selection (ablation) | taxonomy generation · held-out labeling | accuracy: 80.41 -> 84.17 | Improved |
| use GIS instead of center-only selection (ablation) | taxonomy generation · held-out labeling | accuracy: 81.67 -> 84.17 | Improved |
| use GIS instead of confidence-only selection (ablation) | taxonomy generation · held-out labeling | accuracy: 80.83 -> 84.17 | Improved |
| raise K from 24 to 36 (ablation) | GEM + Perf · downstream average | accuracy: 41.21% | Improved |
| raise K from 36 to 48 (ablation) | GEM + Perf · downstream average | accuracy | Hurt |
| use GEM taxonomy instead of WebOrganizer under DoReMi | DoReMi mixing · downstream average | average accuracy: 42.79 -> 43.95 | Improved |
| use GEM taxonomy instead of WebOrganizer under Perf | Perf mixing · downstream average | average accuracy: 44.25 -> 44.79 | Improved |
| use GEM taxonomy instead of WebOrganizer under RegMix | RegMix mixing · downstream average | average accuracy: 40.77 -> 41.45 | Improved |
| use GEM partitions instead of K-Means to distill FastText student | student classifier · test split | accuracy: 72.92 -> 75.13 | Improved |
Closest papers by meaning
- Geometric Constraints for Small Language Models to Understand and Expand Scientific Taxonomies (ICLR 2026 · similarity 0.85)
- MixMin: Finding Data Mixtures via Convex Minimization (ICML 2025 · similarity 0.85)
- Uncovering the Latent Potential of Deep Intermediate Representations (ICML 2026 · similarity 0.84)
- Bridging Compute- and Data-Optimal Pretraining (NeurIPS 2026 · similarity 0.83)
- Nexus: Same Pretraining Loss, Better Downstream Generalization via Common Minima (NeurIPS 2026 · similarity 0.83)
- Data Mixture Optimization: A Multi-fidelity Multi-scale Bayesian Framework (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
