AI Paper Atlas

Research map / Label-efficient and robust learning / Domain adaptation and generalization

Diffract: Spectral View of LLM Domain Adaptation

Nikita Borodin, Maria Krylova, Artem Zabolotnyi, Dmitry Aspisov, Egor Shikov, Nikita Tyuplyaev, Oleg Travkin, Roman Alferov, Dmitry Vinichenko

ICML 2026 · Label-efficient and robust learning › Domain adaptation and generalization · approach: Conditional distribution alignment · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
rewind low-importance heads to pre-train weights (scaled-Frobenius ordering, ~15% heads)math CPT · GSM8K 7Baccuracy: up to +4%Improved
rewind up to 60% of heads using the ordering criterionCPT delta · 7B/13Bquality: up to 60%No clear effect
rewind heads on released OLMo 2 13B CPTOLMo 2 13B · GSM8Kaccuracy: +6.3%Improved
rewind heads on released OLMo 2 32B CPTOLMo 2 32B · GSM8Kaccuracy: up to 3.6%Improved
rewind heads on 13B math CPTOLMo 2 13B · MATH-500accuracy: +2.2%Improved
order heads by difference in scaled Frobenius norms instead of greedyhead rewind · math CPT 1B AUC-diffAUC-diff: 19.1 -> 11Hurt
order heads by difference in scaled Frobenius norms instead of agreement averagehead rewind · math CPT 7B AUC-diffAUC-diff: 4.5 -> 6Improved
order heads by scaled Frobenius norm instead of PL KShead rewind · math CPT 1B AUC-diffAUC-diff: 0.9 -> 11Improved
transplant pre-train singular values into CPT model (ablation)math CPT · GSM8KaccuracyNo clear effect
shuffle only bulk singular values (ablation)trained checkpoint · qualityqualityNo clear effect
shuffle full singular spectrum (ablation)trained checkpoint · qualityquality: drops almost to zeroHurt
truncate 30% of CPT delta singular values (ablation)1B math CPT · GSM8Krelative accuracy: drop of ≈ 10%Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy