Research map / Vision-language and multimodal / Multimodal LLMs
ADEM-VL: Adaptive and Embedded Fusion for Efficient Vision-Language Tuning
Zhiwei Hao, Jianyuan Guo, Li Shen, Yong Luo, Han Hu, Yonggang Wen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add [cls] token to input (ablation) | ADEM-VL baseline · ScienceQA ablation (LLaMA-7B) | accuracy (%): 92.45 -> 92.61 | Improved |
| add SiLU parameter-free cross-attention (ablation) | ADEM-VL · ScienceQA ablation (LLaMA-7B) | accuracy (%): 92.61 -> 93.37 | Improved |
| add multiscale visual prompts (ablation) | ADEM-VL · ScienceQA ablation (LLaMA-7B) | accuracy (%): 93.37 -> 93.47 | Improved |
| add adaptive fusion that drops unimportant visual features (ablation) | ADEM-VL · ScienceQA ablation (LLaMA-7B) | accuracy (%): 93.47 -> 93.85 | Improved |
| use SiLU projection instead of identity (ablation) | parameter-free cross-attention · ScienceQA ablation (LLaMA-7B) | accuracy (%): 92.16 -> 93.85 | Improved |
| use softmax projection instead of SiLU (ablation) | parameter-free cross-attention · ScienceQA ablation (LLaMA-7B) | accuracy (%): 93.85 -> 79.42 | Hurt |
| use ReLU projection instead of SiLU (ablation) | parameter-free cross-attention · ScienceQA ablation (LLaMA-7B) | accuracy (%): 93.85 -> 91.99 | Hurt |
| shift SiLU to be positive (subtract min) (ablation) | parameter-free cross-attention · ScienceQA ablation (LLaMA-7B) | accuracy (%): 93.85 -> 38.58 | Hurt |
| fuse from MLP input to MLP output instead of MHSA(in)->MHSA(out) (ablation) | cross-attention placement · ScienceQA ablation (LLaMA-7B) | accuracy (%): 92.19 -> 93.85 | Improved |
| use average pooling instead of max pooling for multiscale features (ablation) | multiscale visual prompts · ScienceQA ablation (LLaMA-7B) | accuracy (%): 93.55 -> 93.85 | Improved |
| add 256 visual tokens at input stage alongside [cls] (ablation) | ADEM-VL · ScienceQA ablation (LLaMA-7B) | accuracy (%): 93.85 -> 90.17 | Hurt |
| use ADEM-VL instead of LaVIN (13B) | LLaMA-13B captioning · COCO Caption | BLEU-4: 37.8 -> 38.5 | Improved |
Closest papers by meaning
- Florence-VL: Enhancing Vision-Language Models with Generative Vision Encoder and Depth-Breadth Fusion (CVPR 2025 · similarity 0.91)
- Towards General Continuous Memory for Vision-Language Models (NeurIPS 2025 · similarity 0.90)
- VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Tasks (NeurIPS 2024 · similarity 0.90)
- CoVFT: Context-aware Visual Fine-tuning for Multimodal Large Language Models (CVPR 2026 · similarity 0.90)
- LoMo: Local Modality Substitution for Deeper Vision-Language Fusion (NeurIPS 2026 · similarity 0.90)
- AdaptVision: Efficient Vision-Language Models via Adaptive Visual Acquisition (CVPR 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
