Research map / Architectures and efficiency / Pruning and efficient tuning
HiFICL: High-Fidelity In-Context Learning for Multimodal Tasks
Xiaoyu Li, Yuhang Liu, Xuanshuo Kang, Zheng Luo, Fangqi Lou, Xiaohua Wu, Zihan Xiong
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| HiFICL virtual key-value pairs instead of MimIC shift vector | LLaVA-Interleave-7B / Idefics2 · OK-VQA, VQAv2 | VQA accuracy: OK-VQA 52.29 -> 54.19 (LLaVA); VQAv2 69.29 -> 72.08 (Idefics2) | Improved |
| HiFICL (2.2M params) versus LoRA/MimIC on LLaVA | LLaVA-Interleave-7B · VQAv2 | trainable params and accuracy: 17.0M -> 2.2M params; 74.40 -> 74.66 | Improved |
| increase ICL shots from 16 to 32 | Idefics2 ICL · VQAv2 | VQA accuracy: 67.70 -> 50.55 | Hurt |
| add teacher-student alignment to HiFICL (ablation) | HiFICL · VQAv2 | VQA accuracy: 72.08 -> 70.09 | Hurt |
| add low-rank factorization on key matrix (ablation) | HiFICL · OK-VQA | VQA accuracy: 55.72 -> 59.56 | Improved |
| add low-rank factorization on value matrix (ablation) | HiFICL · VQAv2 | VQA accuracy: 69.31 -> 72.08 | Improved |
| add self-attention scaling alpha (non-linear modulation) (ablation) | HiFICL · VQAv2 | VQA accuracy: 70.14 -> 72.08 | Improved |
| teacher-free training instead of teacher (MimIC) paradigm: training cost | HiFICL · VQAv2 training cost | relative FLOPs: 14.3x (MimIC) -> 1x | Improved |
| rank r=16 instead of 8 (ablation) | HiFICL · OK-VQA vs VQAv2 (Idefics2) | accuracy | Mixed |
| HiFICL instead of 8-shot ICL on COCO captions | Idefics2 · COCO hallucination | CHAIRi: 3.9 -> 2.2 | Improved |
Closest papers by meaning
- Make LVLMs Focus: Context-Aware Attention Modulation for Better Multimodal In-Context Learning (AAAI 2026 · similarity 0.88)
- Multimodal Task Vectors Enable Many-Shot Multimodal In-Context Learning (NeurIPS 2024 · similarity 0.88)
- AIM: Let Any Multimodal Large Language Models Embrace Efficient In-Context Learning (AAAI 2025 · similarity 0.88)
- Mimic In-Context Learning for Multimodal Tasks (CVPR 2025 · similarity 0.87)
- Beyond task performance: evaluating and reducing the flaws of large multimodal models with in-context-learning (ICLR 2024 · similarity 0.87)
- VL-ICL Bench: The Devil in the Details of Multimodal In-Context Learning (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
