Research map / Large language models / Reasoning and chain of thought

Scaling Laws Meet Model Architecture: Toward Inference-Efficient LLMs

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Efficient LLM serving · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes an architecture-aware conditional scaling law extending Chinchilla with hidden size, mlp-to-attention ratio, and GQA, and uses it to search for inference-efficient, accurate LLM architectures (Panda and Surefire models at 1B and 3B).

Problem

Existing scaling laws ignore inference cost, and prior architecture-aware scaling law work only considers aspect ratio and fixes depth; the trade-off between accuracy and inference efficiency under fixed parameter and token budgets is underexplored.

Approach

A two-step conditional scaling law: take the Chinchilla optimal loss as a reference, then calibrate it multiplicatively or additively with separable U-shaped terms in hidden size normalized by sqrt(N) and in mlp-to-attention ratio; then search architectures under a loss constraint and locally search GQA for throughput.

Key results

• Panda-1B outperforms LLaMA-3.2-1B by 2.1% average accuracy on nine downstream tasks under the same 100B-token training budget.
• Surefire-1B reaches up to 42% higher inference throughput (vLLM, A100) than LLaMA-3.2-3B-scale comparisons, and up to 47% higher throughput with SGLang on H200.
• Panda-3B outperforms LLaMA-3.2-3B by 0.6% average accuracy; the conditional scaling law gives low MSE and high Spearman correlation when predicting loss at larger scales.

Surprising findings

• Fitting the scaling law with 1B variants predicts 3B behavior better than fitting with smaller 80M-297M models, suggesting coefficients shift with model size.
• Including outlier mlp-to-attention ratios (below 0.5 or above 5) degrades Spearman correlation of the fitted law, so they are excluded.

Limitations

• Evaluation does not extend to 7B models due to resource and time limits.
• Analysis restricted to dense models; no scaling law established for MoE architectures.
• Only pretraining studied; effects under post-training are unknown.

Experimental setup

Datasets: Dolma-v1.7

Benchmarks: ARC-Easy ARC-Challenge LAMBADA HellaSwag OpenBookQA PIQA SciQ WinoGrande CoQA WikiText perplexity

Metrics: zero-shot accuracy average accuracy training loss MSE Spearman correlation inference throughput (tokens/s)

Baselines: LLaMA-3.2-1B LLaMA-3.2-3B OLMo-2-1B Qwen2.5-3B LLaMA-3.2-1B-HF LLaMA-3.2-3B-HF

Builds on: Chinchilla scaling laws LLaMA-3.2 Qwen3 vLLM

Closest papers by meaning

• Scaling Inference-Efficient Language Models (ICML 2025 · similarity 0.95)
• Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for LLM Problem-Solving (ICLR 2025 · similarity 0.93)
• Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws (ICML 2024 · similarity 0.92)
• LLMs on the Line: Data Determines Loss-to-Loss Scaling Laws (ICML 2025 · similarity 0.91)
• Scaling Laws for Precision (ICLR 2025 · similarity 0.91)
• A Hitchhiker's Guide to Scaling Law Estimation (ICML 2025 · similarity 0.91)