AI Paper Atlas

Research map / Large language models / Instruction and fine-tuning

Theoretical Perspectives on Data Quality and Synergistic Effects in Pre- and Post-Training Reasoning Models

Adel Javanmard, Baharan Mirzasoleiman, Vahab Mirrokni

ICML 2026 · Large language models › Instruction and fine-tuning · approach: Fine-tuning generalization and delta compression · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
post-train SFT on examples aligned with the adaptation shift (smaller theta, harder for pretrained model) (ablation)LSA transformer SFT · in-context weight prediction (d=400, m=200)post-test errorImproved
increase number of SFT prompts B or prompt length n beyond an intermediate value, with interference (r>0) (ablation)LSA transformer SFT · in-context weight prediction (d=400, m=200)post-test errorHurt
increase SFT prompts B with no interference (r=0) (ablation)LSA transformer SFT · in-context weight predictionpost-test errorMixed
increase SFT sample size B or context length n (ablation)GPT-2 (9.5M) SFT · in-context weight predictiontest lossMixed
increase post-training CoT length k (ablation)GPT-2 (9.5M) SFT · in-context weight predictiontest lossNo clear effect
increase outcome-supervision sample size B and context length n (ablation)GPT-2 (9.5M) OS · in-context weight predictiontest lossImproved
increase post-training CoT length k (ablation)GPT-2 (9.5M) OS · in-context weight predictiontest lossHurt
weaken interference by raising r from 0.01 to 0.1 (ablation)LSA transformer SFT · in-context weight predictionpost-test errorImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy