AI Paper Atlas

Research map / Large language models / Instruction and fine-tuning

Fresh in memory: Training-order recency is linearly encoded in language model activations

Dmitrii Krasheninnikov, Richard E. Turner, David Krueger

ICLR 2026 · Large language models › Instruction and fine-tuning · approach: Efficient LLM fine-tuning systems · 5 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
fine-tune 30 extra epochs on shuffled mixed data from all stages (ablation)Llama-3.2-1B probes · D1 vs D6 probing (synthetic)probe accuracy (%): 90 -> 63Hurt
fine-tune 30 extra epochs on shuffled mixed data from all stages (ablation)Llama-3.2-1B probes · D1 vs D6 probing (natural)probe accuracy (%): 80 -> 60Hurt
use LoRA on Llama-3.1-8B instead of full fine-tuning (ablation)Llama-3.1-8B · D1 vs D6 probingprobe accuracy (%): ~85No clear effect
use 1-epoch stages with 5x more unique examples instead of 5 epochs (ablation)Llama-3.2-1B natural data · D1 vs D6 probingprobe accuracy (%)No clear effect
re-expose one earlier stage as a 7th fine-tuning stage (ablation)Llama-3.2-1B · activation centroid ordercentroid positionImproved
train one stage for 15 epochs instead of 5 (ablation)Llama-3.2-1B · activation centroid ordercentroid gapImproved
balance joint distribution of six activation statistics between D1 and D2 (ablation)Llama-3.2-1B probes · D1 vs D2 probingprobe accuracy (%)No clear effect
balance joint distribution of seven logit statistics (ablation)Llama-3.2-1B probes · D1 vs D2 probingprobe accuracy (%)No clear effect
balance forward-looking multi-token statistics (ablation)Llama-3.2-1B probes · D1 vs D2 probingprobe accuracy (%)Mixed
fine-tune to report training stage explicitlyLlama-3.2-1B (D1 then D2) · held-out aliasesaccuracy (%): 79.8Improved
probe specific training datapoints instead of held-out entitiesLlama-3.2-1B · question-level stage probingprobe accuracy (%): over 90 -> ~60Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy