AI Paper Atlas

Research map / Large language models / Instruction and fine-tuning

SHINE: A Scalable In-Context Hypernetwork for Mapping Context to LoRA in a Single Pass

Yewei Liu, Xiyuan Wang, Yansheng Mao, Yoav Gelberg, Haggai Maron, Muhan Zhang

ICML 2026 · Large language models › Instruction and fine-tuning · approach: Low-rank adaptation variants · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
generate LoRA with SHINE hypernetwork instead of SFT on the contextQwen3-8B · MS MARCO MQAanswer F1: 33.0 -> 55.6Improved
generate LoRA with SHINE instead of no context (Naive)Qwen3-8B · MS MARCO MQAanswer F1: 23.2 -> 55.6Improved
replace SFT per-context training with one hypernetwork forward passQwen3-8B · MS MARCO MQAamortizable time (s): 29.3 -> 0.3Improved
use SHINE instead of In-Context promptingQwen3-8B · MS MARCO MQAanswer F1: 69.4 -> 55.6Hurt
use SHINE instead of Generative AdapterQwen3-8B · MS MARCO V2answer F1: 27.9 -> 40.1Improved
use SHINE instead of Generative AdapterQwen3-8B · SQuADanswer F1: 70.3 -> 63.6Hurt
use SHINE instead of TTT method SEALQwen3-8B · SQuAD (n=200)score: 58.2 -> 63.6Improved
train SHINE on full MQA data instead of half (ablation)SHINE · SQuAD-2kanswer F1: 33.8 -> 37.5Improved
train SHINE on full 1QA data instead of half (ablation)SHINE · SQuADanswer F1: 62.2 -> 63.6Improved
train SHINE on full MQA data instead of half (ablation)SHINE · MS MARCO V2answer F1: 40.8 -> 40.1Hurt
increase M2P Transformer layers from 4 to 6 (ablation)SHINE · Wikitext-2 validationPPL: 1.88 -> 1.79Improved
increase meta LoRA rank from 4 to 128 with generated rank 8 (vs 8/8) (ablation)SHINE · Wikitext-2 validationPPL: 1.63 -> 1.32Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy