Research map / Large language models / Language, safety and interpretability

A Theory of Training Profit-Optimal LLMs

NeurIPS 2026 · Large language models › Language, safety and interpretability · approach: LLM unlearning and benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Builds an economic model of a monopolist LLM firm combining Leontief scaling laws with quasilinear inverse demand, and derives profit-optimal training expenditure in compute-bound and data-bound regimes, then compares to empirical trends.

Problem

Whether scaling LLM quality (via parameters and tokens) yields enough revenue to offset training and inference costs, i.e., what profit-optimal LLM training looks like as hardware and algorithmic efficiency change.

Approach

Couples neural scaling laws (Leontief/min form for quality) with microeconomic quasilinear inverse demand where consumers have reservation qualities following a power law, yielding closed-form asymptotic bounds on profit-optimal model size and training expenditure as functions of hardware efficiency E and efficiency parameters a, b.</ask>

Key results

• In the compute-bound regime, optimal training expenditure scales as E^{(1-alpha*gamma)/(1+alpha*gamma)}; with alpha~0.3 and gamma=-1, the bound is about 7.59^t, consistent with the observed ~5^t growth in training compute.
• In the data-bound regime, optimal training expenditure scales roughly as D^{1+beta/alpha} / E, so it grows quadratically with data and decreases with hardware efficiency.
• Break-even gamma-hat is approximately -0.77; with gamma=0 current training compute growth exceeds the profit-optimal bound (4.11^t vs 5^t).

Surprising findings

• Hardware efficiency improvements reduce profit-optimal training expenditure in the data-bound regime.
• Under the model, current training expenditure growth exceeds profit-optimal levels for most gamma values, except near gamma=-1.

Limitations

• Assumes a monopoly firm; competition not modeled.
• Leontief scaling law approximates Chinchilla scaling; quality definition is idealized.
• Assumes inverse demand is diminishing (gamma > -1) in the main analysis; upper bounds may not be tight.

Experimental setup

Datasets: Epoch.ai trends dashboard (compute, hardware efficiency, algorithmic efficiency growth rates)

Metrics: asymptotic scaling exponents of optimal model size, data, an break-even demand elasticity gamma-hat

Baselines: Chinchilla scaling law

Builds on: Chinchilla scaling law compute-optimal training framework quantization model of demand

Closest papers by meaning

• Unveiling the Secret Recipe: A Guide For Supervised Fine-Tuning Small LLMs (ICLR 2025 · similarity 0.90)
• Scaling Laws Meet Model Architecture: Toward Inference-Efficient LLMs (ICLR 2026 · similarity 0.90)
• Cost-of-Pass: An Economic Framework for Evaluating Language Models (ICLR 2026 · similarity 0.90)
• LLMs on the Line: Data Determines Loss-to-Loss Scaling Laws (ICML 2025 · similarity 0.89)
• Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for LLM Problem-Solving (ICLR 2025 · similarity 0.89)
• Compute-Optimal LLMs Provably Generalize Better with Scale (ICLR 2025 · similarity 0.89)