Research map / Vision-language and multimodal / Cross-modal learning
Multi-modal object re-identification
Selects tokens and decouples features across RGB, infrared and text for re-identification.
40 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2019–2026) · an approach in Cross-modal learning, keyword cluster “fusion · missing · multi modal”
State History Benchmarks Tried Checklist Preprints Who Export
Where it stands
Still moving The best reported result is still improving on 3 of 3 tracked benchmarks.
11 papers in 2023–24 and 26 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 3.8% to 3.3%.
Open questions the data points to
- RGBNT201 (mAP): best went from 66.5 (up to 2024) to 81.2 in 2025 — UGG-ReID: Uncertainty-Guided Graph Model for Multi-Modal Object Re-Identification (NeurIPS 2025 · 0 citations)
- MSVR310 (mAP): best went from 39 (up to 2024) to 60.1 in 2025 — UGG-ReID: Uncertainty-Guided Graph Model for Multi-Modal Object Re-Identification (NeurIPS 2025 · 0 citations)
- RGBNT100 (mAP): best went from 82.1 (up to 2024) to 88.2 in 2025 — Multi-Modal Object Re-identification via Sparse Mixture-of-Experts (ICML 2025)
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2019 · A Dataset and Benchmark for Large-Scale Multi-Modal Face Anti-Spoofing (CVPR 2019)
- 2021 · Robust Multi-Modality Person Re-identification (AAAI 2021 · 61 citations)
- 2022 · Interact, Embed, and EnlargE: Boosting Modality-Specific Representations for Multi-Modal Person Re-identification (AAAI 2022 · 54 citations)
- 2023 · Towards Modality-Agnostic Person Re-Identification With Descriptive Query (CVPR 2023 · 47 citations)
- 2024 · Magic Tokens: Select Diverse Tokens for Multi-modal Object Re-Identification (CVPR 2024 · 58 citations)
- 2025 · DeMo: Decoupled Feature-Based Mixture of Experts for Multi-Modal Object Re-Identification (AAAI 2025 · 30 citations)
- 2026 · Signal: Selective Interaction and Global-local Alignment for Multi-Modal Object Re-Identification (AAAI 2026 · 2 citations)
Earliest papers here
- A Dataset and Benchmark for Large-Scale Multi-Modal Face Anti-Spoofing (CVPR 2019)
- Robust Multi-Modality Person Re-identification (AAAI 2021 · 61 citations)
- Interact, Embed, and EnlargE: Boosting Modality-Specific Representations for Multi-Modal Person Re-identification (AAAI 2022 · 54 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
RGBNT201 · mAP (higher is better)
8 papers, 2021–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2021 | 31.76 | Robust Multi-Modality Person Re-identification (AAAI 2021 · 61 citations) |
| 2022 | 46.42 | Interact, Embed, and EnlargE: Boosting Modality-Specific Representations for Multi-Modal Person Re-identification (AAAI 2022 · 54 citations) |
| 2024 | 66.5 | Magic Tokens: Select Diverse Tokens for Multi-modal Object Re-Identification (CVPR 2024 · 58 citations) |
| 2025 | 81.2 | UGG-ReID: Uncertainty-Guided Graph Model for Multi-Modal Object Re-Identification (NeurIPS 2025 · 0 citations) |
MSVR310 · mAP (higher is better)
6 papers, 2024–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2024 | 39 | Magic Tokens: Select Diverse Tokens for Multi-modal Object Re-Identification (CVPR 2024 · 58 citations) |
| 2025 | 60.1 | UGG-ReID: Uncertainty-Guided Graph Model for Multi-Modal Object Re-Identification (NeurIPS 2025 · 0 citations) |
RGBNT100 · mAP (higher is better)
5 papers, 2024–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2024 | 82.1 | Magic Tokens: Select Diverse Tokens for Multi-modal Object Re-Identification (CVPR 2024 · 58 citations) |
| 2025 | 88.2 | Multi-Modal Object Re-identification via Sparse Mixture-of-Experts (ICML 2025) |
What has been tried
25 papers here report 311 measured changes (a component added, removed or swapped, compared with a baseline): helped 82%, no clear effect 3%, hurt 12%, mixed 3%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Add HMA | RGBNT201 · mAP | 54.0 -> 60.7 | Helped | Magic Tokens: Select Diverse Tokens for Multi-modal Object Re-Identification (CVPR 2024 · 58 citations) |
| Add U-Adapter | Protocol 1 average (four cross-dataset sub-protocols) · HTER (%) | 31.14 -> 27.38 | Helped | Suppress and Rebalance: Towards Generalized Multi-Modal Face Anti-Spoofing (CVPR 2024 · 36 citations) |
| Add fine-granularity parallel contrast SP-W to SI-T | Occluded-CUHK-PEDES · R@1 | 57.33 -> 62.23 | Helped | Text-Based Occluded Person Re-identification via Multi-Granularity Contrastive Consistency Learning (AAAI 2024 · 31 citations) |
| Add PIFE to the baseline | RGBNT201 · mAP | 70.7 -> 73.0 | Helped | DeMo: Decoupled Feature-Based Mixture of Experts for Multi-Modal Object Re-Identification (AAAI 2025 · 30 citations) |
| Add PFA to frozen CLIP | RGBNT201 · mAP | 69.4 -> 73.2 | Helped | MambaPro: Multi-Modal Object Re-identification with Mamba Aggregation and Synergistic Prompt (AAAI 2025 · 23 citations) |
| add text guidance | RGBNT201 · mAP | 74.5 -> 80.2 | Helped | IDEA: Inverted Text with Cooperative Deformable Aggregation for Multi-modal Object Re-Identification (CVPR 2025 · 16 citations) |
| add semantic filtered fusion (SFF) | Tri-SYSU-MM01 · Rank-1 | 72.52 -> 77.00 | Helped | Empowering Visible-Infrared Person Re-Identification with Large Foundation Models (NeurIPS 2024 · 15 citations) |
| add CMPS to UAP-Retrieval | RegDB and SYSU · mAP | 6.87 -> 3.98; 4.76 -> 3.42 | Helped | Cross-Modality Perturbation Synergy Attack for Person Re-identification (NeurIPS 2024 · 14 citations) |
| add ComPA to the CLIP baseline | MIMIC-X · CMC-R1 | 92.29 -> 96.60 | Helped | Towards All-in-One Medical Image Re-Identification (CVPR 2025 · 8 citations) |
| Remove the soft-biometric attributes | Ethnic, FaceScrub, IMDB, and Cross-Modal DB; intra- and cross-modality recognition · rank-1 accuracy | Hurt | Flexible Biometrics Recognition: Bridging the Multimodality Gap through Attention Alignment and Prompt Tuning (CVPR 2024 · 7 citations) | |
| Trigger size increased from 4x4 to 32x32 on backdoored ViT-B/16 | ViT-B/16, basketball target · AI | AI stays above threshold 3 (4.2 at 4x4, 3.53 at 32x32); top text basketball in most cases | No clear effect | SEER: Backdoor Detection for Vision-Language Models through Searching Target Text and Image Trigger Jointly (AAAI 2024 · 6 citations) |
| Add the disorder local learning module to the global architecture | CUHK-PEDES · Rank-1 accuracy | +4.37% | Helped | Multi-Modal Disordered Representation Learning Network for Description-Based Person Search (AAAI 2024 · 5 citations) |
What reviewers will expect
Datasets and baselines named in the 17 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- RGBNT201 · 6 of 17 papers
- RGBNT100 · 6 of 17 papers
- MSVR310 · 6 of 17 papers
- CUHK-PEDES · 2 of 17 papers
- ICFG-PEDES · 2 of 17 papers
- SYSU-MM01 · 2 of 17 papers
Baselines
- TOP-ReID · 6 of 17 papers
- EDITOR · 6 of 17 papers
- RSCNet · 6 of 17 papers
- HTT · 6 of 17 papers
- UniCat · 6 of 17 papers
- IEEE · 5 of 17 papers
- PFNet · 4 of 17 papers
- TransReID · 4 of 17 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 37 papers; from OpenAlex and the papers' affiliations).
- Xiamen University · 5 papers
- Anhui University · 4 papers
- Dalian University of Technology · 3 papers
- Wuhan University · 3 papers
- Shanghai AI Laboratory · 3 papers
- Beijing University of Posts and Telecommunications · 2 papers
- Shenzhen University · 2 papers
- Tencent · 2 papers
- Northeastern University · 2 papers
- National University of Singapore · 2 papers
Use it
Download BibTeX of 24 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Multimodal fusion with missing modalities (208 papers)
- Multi-modality image fusion (194 papers)
- Medical multimodal fusion (166 papers)
- Multimodal sentiment analysis (106 papers)
Elsewhere in the topic
- Unimodal-to-multimodal representation transfer (145 papers)
- Probabilistic embeddings for cross-modal retrieval (108 papers)
- Contrastive vision-language alignment (104 papers)
- Audio-visual speech separation (76 papers)
- Audio-visual source localization (67 papers)
- Cross-modal hashing (45 papers)
- Self-supervised audio-visual representation learning (44 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).