Research map / Large language models / Instruction and fine-tuning
SHINE: A Scalable In-Context Hypernetwork for Mapping Context to LoRA in a Single Pass
Yewei Liu, Xiyuan Wang, Yansheng Mao, Yoav Gelberg, Haggai Maron, Muhan Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| generate LoRA with SHINE hypernetwork instead of SFT on the context | Qwen3-8B · MS MARCO MQA | answer F1: 33.0 -> 55.6 | Improved |
| generate LoRA with SHINE instead of no context (Naive) | Qwen3-8B · MS MARCO MQA | answer F1: 23.2 -> 55.6 | Improved |
| replace SFT per-context training with one hypernetwork forward pass | Qwen3-8B · MS MARCO MQA | amortizable time (s): 29.3 -> 0.3 | Improved |
| use SHINE instead of In-Context prompting | Qwen3-8B · MS MARCO MQA | answer F1: 69.4 -> 55.6 | Hurt |
| use SHINE instead of Generative Adapter | Qwen3-8B · MS MARCO V2 | answer F1: 27.9 -> 40.1 | Improved |
| use SHINE instead of Generative Adapter | Qwen3-8B · SQuAD | answer F1: 70.3 -> 63.6 | Hurt |
| use SHINE instead of TTT method SEAL | Qwen3-8B · SQuAD (n=200) | score: 58.2 -> 63.6 | Improved |
| train SHINE on full MQA data instead of half (ablation) | SHINE · SQuAD-2k | answer F1: 33.8 -> 37.5 | Improved |
| train SHINE on full 1QA data instead of half (ablation) | SHINE · SQuAD | answer F1: 62.2 -> 63.6 | Improved |
| train SHINE on full MQA data instead of half (ablation) | SHINE · MS MARCO V2 | answer F1: 40.8 -> 40.1 | Hurt |
| increase M2P Transformer layers from 4 to 6 (ablation) | SHINE · Wikitext-2 validation | PPL: 1.88 -> 1.79 | Improved |
| increase meta LoRA rank from 4 to 128 with generated rank 8 (vs 8/8) (ablation) | SHINE · Wikitext-2 validation | PPL: 1.63 -> 1.32 | Improved |
Closest papers by meaning
- Doc-to-LoRA: Learning to Instantly Internalize Contexts (ICML 2026 · similarity 0.91)
- Text-to-LoRA: Instant Transformer Adaption (ICML 2025 · similarity 0.88)
- LoRA-Gen: Specializing Large Language Model via Online LoRA Generation (ICML 2025 · similarity 0.87)
- Generative Adapter: Contextualizing Language Models in Parameters with A Single Forward Pass (ICLR 2025 · similarity 0.87)
- LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models (ICLR 2024 · similarity 0.87)
- LoRASuite: Efficient LoRA Adaptation Across Large Language Model Upgrades (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
