Research map / Vision-language and multimodal / Cross-modal learning

Multi-modality image fusion

Fuses RGB, infrared, event and other modalities with transformers and semantic priors.

194 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Cross-modal learning, keyword cluster “fusion · missing · multi modal”

State History Benchmarks Tried Baselines Checklist Preprints Who Export

How it got here

Most cited paper of each year

• 2016 · Discriminative Multi-Modal Feature Fusion for RGBD Indoor Scene Recognition (CVPR 2016 · 115 citations)
• 2017 · AMC: Attention guided Multi-modal Correlation Learning for Image Search (CVPR 2017 · 37 citations)
• 2018 · Robust Collaborative Discriminative Learning for RGB-Infrared Tracking (AAAI 2018 · 78 citations)
• 2019 · Selective Sensor Fusion for Neural Visual-Inertial Odometry (CVPR 2019 · 167 citations)
• 2020 · Unsupervised Multi-Modal Image Registration via Geometry Preserving Image-to-Image Translation (CVPR 2020 · 175 citations)
• 2021 · Encoder Fusion Network With Co-Attention Embedding for Referring Image Segmentation (CVPR 2021 · 179 citations)
• 2022 · VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Experts (NeurIPS 2022 · 285 citations)
• 2023 · Delivering Arbitrary-Modal Semantic Segmentation (CVPR 2023 · 185 citations)
• 2024 · Equivariant Multi-Modality Image Fusion (CVPR 2024 · 233 citations)
• 2025 · DCEvo: Discriminative Cross-Dimensional Evolutionary Learning for Infrared and Visible Image Fusion (CVPR 2025 · 76 citations)
• 2026 · Text-Guided Channel Perturbation and Pre-Trained Knowledge Integration for Unified Multi-Modality Image Fusion (AAAI 2026 · 7 citations)

Earliest papers here

• Discriminative Multi-Modal Feature Fusion for RGBD Indoor Scene Recognition (CVPR 2016 · 115 citations)
• AMC: Attention guided Multi-modal Correlation Learning for Image Search (CVPR 2017 · 37 citations)
• Robust Collaborative Discriminative Learning for RGB-Infrared Tracking (AAAI 2018 · 78 citations)

Benchmark records

The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.

What has been tried

100 papers here report 1,116 measured changes (a component added, removed or swapped, compared with a baseline): helped 79%, no clear effect 5%, hurt 14%, mixed 2%. Papers mostly publish changes that helped, so "helped" is over-represented.

Ablations in recent papers

Baseline numbers that disagree

The same baseline on the same benchmark and metric, reported more than 5% apart by different papers here. Training budgets, data splits and reimplementations differ, so state which number you compare with.

What reviewers will expect

Datasets and baselines named in the 40 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.

Recent preprints

Who publishes on it

Institutions on the most papers of this approach since 2023 (of 167 papers; from OpenAlex and the papers' affiliations).

• Tsinghua University · 10 papers
• Peking University · 6 papers
• Wuhan University · 6 papers
• University of Science and Technology of China · 5 papers
• Zhejiang University · 5 papers
• Southeast University · 5 papers
• Tianjin University · 4 papers
• University of Surrey · 4 papers
• Tencent · 4 papers
• Shanghai Jiao Tong University · 4 papers

Use it

Download BibTeX of 34 key papersOpen the topic in the interactive map

Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.

Related approaches

Same keyword cluster

• Multimodal fusion with missing modalities (208 papers)
• Medical multimodal fusion (166 papers)
• Multimodal sentiment analysis (106 papers)
• Multi-modal object re-identification (40 papers)

Elsewhere in the topic

• Unimodal-to-multimodal representation transfer (145 papers)
• Probabilistic embeddings for cross-modal retrieval (108 papers)
• Contrastive vision-language alignment (104 papers)
• Audio-visual speech separation (76 papers)
• Audio-visual source localization (67 papers)
• Cross-modal hashing (45 papers)
• Self-supervised audio-visual representation learning (44 papers)

Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).