AI Paper Atlas

Research map / Large language models / Instruction and fine-tuning

Towards Resource-Efficient LLMs: End-to-End Energy Accounting of Distillation Pipelines

Katherine Lambert, Sasha Luccioni

ICML 2026 · Large language models › Instruction and fine-tuning · approach: Fine-tuning generalization and delta compression · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace baseline SFT with logit KD from 32B teacherOLMo-2 7B student · TULU-3Total energy (kWh): 19.50 -> 28.40Hurt
replace baseline SFT with logit KDOLMo-2 7B student · TULU-3Quality Q: 0.90 -> 0.78Hurt
replace baseline SFT with synthetic SFTOLMo-2 13B student · TULU-3Quality Q: 0.99 -> 0.85Hurt
replace baseline SFT with KD or synthetic SFTOLMo-2 1B student · TULU-3Total energy (kWh): 7.00 -> 16.90Hurt
replace baseline SFT with KD or synthetic SFTOLMo-2 1B student · TULU-3Quality Q: 0.69 -> 0.70Improved
scale student from 1B to 13B (ablation)Baseline SFT · TULU-3Total energy (kWh): 7.00 -> 34.60Mixed
scale student from 1B to 13B (ablation)Baseline SFT · TULU-3Quality Q: 0.69 -> 0.99Improved
reuse cached teacher artifacts across N students (ablation)KD / synthetic SFT · AmortizationEnergy per student: break-even N approx 10 (KD 1B)Improved
use distillation (KD) for student training stageOLMo-2 13B student · TULU-3Student training energy (kWh): 33.15 -> 30.05Improved
increase KD temperature T (ablation)KD 7B student · TULU-3 sweepQualityNo clear effect
increase KD loss weight alpha (ablation)KD students · TULU-3 sweepQualityImproved
increase max_new_tokens for synthetic generation (ablation)Synthetic SFT 7B · TULU-3 sweepQualityImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy