AI Paper Atlas

Research map / Architectures and efficiency / Mixture of experts

Capacity-Aware Mixture Law Enables Efficient LLM Data Optimization

Jingwei Li, Xinran Gu, Jingzhao Zhang

ICML 2026 · Architectures and efficiency › Mixture of experts · approach: Efficient MoE language model systems · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
model the validation loss with a capacity-aware joint law of model size and mixture (CAMEL) instead of separable lawsCAMEL vs DML/SODM · 7 benchmarks, Balanced objective, 55B-A1.2Bweighted average score: 0.526 -> 0.532 (vs SODM); 0.519 (DML)Improved
use CAMEL mixture instead of human-designed mixtureCAMEL · Balanced objective weighted averageweighted average score: 0.496 -> 0.532Improved
use CAMEL mixture instead of model-size-agnostic mixtureCAMEL · Balanced objective weighted averageweighted average score: 0.514 -> 0.532Improved
use CAMEL for Math specialized targetCAMEL vs SODM · Math specialized weighted averageweighted average score: 0.494 -> 0.497Improved
use CAMEL for Code specialized targetCAMEL vs SODM · Code specialized weighted averageweighted average score: 0.520 -> 0.528Improved
use CAMEL for Knowledge specialized targetCAMEL vs SODM · Knowledge specialized weighted averageweighted average score: 0.559 -> 0.565Improved
use hourglass sampling of (model size, mixture) pairs instead of rectangle allocation (ablation)CAMEL fitting · extrapolation error under fixed computeprediction errorImproved
compare triangle, inverted triangle and diamond sampling against hourglass (ablation)CAMEL fitting · extrapolation error under fixed computeprediction errorHurt
vary the number of intrinsic domains k (ablation)CAMEL · loss predictionprediction errorNo clear effect
use CAMEL instead of DML on MATHCAMEL vs DML · MATH (Balanced mixture)accuracy: 0.393 -> 0.372Hurt
use CAMEL instead of SODM on GSM8KCAMEL vs SODM · GSM8K (Balanced mixture)accuracy: 0.705 -> 0.693Hurt
fit loss-to-benchmark logistic law on validation lossesloss-to-benchmark law · 14 benchmarksbenchmark prediction errorImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy