Research map / Vision-language and multimodal / CLIP and zero-shot
Expanding Large Pre-Trained Unimodal Models With Multimodal Information Injection for Image-Text Multimodal Classification
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Expand the unimodal backbones with MI2P and late cross-attention fusion | DenseNet and BERT · CrisisMMD Task 1 | accuracy (%): 88.4 -> 92.0 | Helped |
| Expand the unimodal backbones with MI2P and late cross-attention fusion | DenseNet and BERT · CrisisMMD Task 2 | Macro F1-score (%): 67.8 -> 85.6 | Helped |
| Expand the unimodal backbones with MI2P and late cross-attention fusion | DenseNet and BERT · CrisisMMD Task 3 | weighted F1-score (%): 69.7 -> 75.7 | Helped |
| Expand the unimodal backbones with MI2P and late cross-attention fusion | DenseNet and BERT · MM-IMDB | Micro F1-score (%): 69.3 -> 70.8 | Helped |
| Use MI2P-expanded models with feature concatenation | DenseNet and BERT · Food101 | accuracy (%): 92.9 -> 94.7 | Helped |
| Add MI2P at dense block 1 (ablation) | DenseNet · CrisisMMD Task 2 | accuracy (%): 83.4 -> 88.1 | Helped |
| Add MI2P at dense block 2 (ablation) | DenseNet · CrisisMMD Task 2 | accuracy (%): 83.4 -> 87.3 | Helped |
| Add MI2P at dense block 3 (ablation) | DenseNet · CrisisMMD Task 2 | accuracy (%): 83.4 -> 87.9 | Helped |
| Add MI2P at dense block 4 (ablation) | DenseNet · CrisisMMD Task 2 | accuracy (%): 83.4 -> 88.4 | Helped |
| Attach MI2P to dense blocks 1–4 rather than use unmodified DenseNet (ablation) | DenseNet · CrisisMMD Task 2 | accuracy (%): 83.4 -> 90.7 | Helped |
| Add MI2P at BERT layers 1–3 (ablation) | BERT · CrisisMMD Task 2 | accuracy (%): 86.1 -> 88.1 | Helped |
| Add MI2P at BERT layers 4–6 (ablation) | BERT · CrisisMMD Task 2 | accuracy (%): 86.1 -> 87.5 | Helped |
Closest papers by meaning
- CSA: Data-efficient Mapping of Unimodal Features to Multimodal Features (ICLR 2025 · similarity 0.85)
- VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks (ICLR 2025 · similarity 0.85)
- Beyond Modality Fusion: Deep Ensembles for Multimodal Classification (NeurIPS 2026 · similarity 0.85)
- Unleashing the Intrinsic Visual Representation Capability of Multimodal Large Language Models (CVPR 2026 · similarity 0.85)
- Multimodal Representation Learning by Alternating Unimodal Adaptation (CVPR 2024 · similarity 0.85)
- MANZANO: A Simple and Scalable Unified Multimodal Model with a Hybrid Vision Tokenizer (ICLR 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.