Research map / Vision-language and multimodal / Cross-modal learning
Magic Tokens: Select Diverse Tokens for Multi-modal Object Re-Identification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EDITOR, a shared-ViT framework that selects object-centric tokens via spatial attention and Haar-wavelet frequency cues, then aggregates them with masked attention and adds background-consistency and object-centric losses for multi-modal (RGB/NIR/TIR) object ReID.
Problem
Multi-modal object re-identification is hurt by irrelevant background tokens and by large modality gaps when features from all image regions are aggregated across RGB, NIR and TIR inputs.
Approach
Select diverse object-centric tokens per instance using the union of head-level and modality-level spatial attention indices plus frequency-based (DHWT) salient tokens, aggregate them hierarchically with masked Transformer blocks, and constrain background tokens across modalities with a consistency loss and features with per-ID centers.
Key results
- On RGBNT201, EDITOR (B) reaches 65.7% mAP and 68.8% R-1, versus 64.6% mAP for TOP-ReID (B).
- On RGBNT100, EDITOR (B) reaches 82.1% mAP, 0.9 points above TOP-ReID (B) at 81.2%.
- On MSVR310, EDITOR (B) reaches 39.0% mAP, versus 35.9% for TOP-ReID (B) and 36.4% for CCNet.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use EDITOR (A) instead of TOP-ReID (A) | Multi-modal person ReID · RGBNT201 | mAP: 72.3 -> 66.5 | Hurt |
| Use EDITOR (B) instead of TOP-ReID (B) | Multi-modal person ReID · RGBNT201 | mAP: 64.6 -> 65.7 | Helped |
| Use EDITOR (B) instead of TOP-ReID (B) | Multi-modal vehicle ReID · RGBNT100 | mAP: 81.2 -> 82.1 | Helped |
| Use EDITOR (B) instead of CCNet | Multi-modal vehicle ReID · MSVR310 | mAP: 36.4 -> 39.0 | Helped |
| Add HMA (ablation) | ViT-B/16 baseline · RGBNT201 | mAP: 54.0 -> 60.7 | Helped |
| Add SFTS (ablation) | Baseline + HMA · RGBNT201 | mAP: 60.7 -> 62.2 | Helped |
| Add BCC (ablation) | Baseline + HMA + SFTS · RGBNT201 | mAP: 62.2 -> 65.2 | Helped |
| Add OCFR (ablation) | Baseline + HMA + SFTS · RGBNT201 | mAP: 62.2 -> 64.8 | Helped |
| Remove token selection (ablation) | HMA model · RGBNT201 | mAP: 62.2 -> 60.7 | Hurt |
| Use modality separation instead of modality union (ablation) | Token selection and aggregation · RGBNT201 | mAP: 62.2 -> 57.7 | Hurt |
| Use modality union instead of separation (ablation) | Token selection and aggregation · RGBNT201 | mAP: 57.7 -> 62.2 | Helped |
| Use spatial-based selection with head union instead of modality-only selection (ablation) | SFTS · RGBNT201 | mAP: 64.2 -> 65.0 | Helped |
Surprising findings
- Modality-separated token selection performed worse than no selection (57.7 vs 60.7 mAP on RGBNT201), while modality-level union helped.
Limitations
- TOP-ReID (A) outperforms EDITOR (A) on RGBNT201 mAP (72.3 vs 66.5).
- Gains over the strongest baseline on RGBNT100 are small (0.9 mAP).
Experimental setup
Datasets: RGBNT201 RGBNT100 MSVR310 ImageNet
Benchmarks: RGBNT201 RGBNT100 MSVR310
Metrics: mAP CMC Rank-1 Rank-5 Rank-10
Baselines: TOP-ReID IEEE UniCat PHT CCNet TransReID PCB AGW DENet PFNet
Builds on: TransReID TransFG DynamicViT TOP-ReID Vision Transformer
Closest papers by meaning
- TOP-ReID: Multi-Spectral Object Re-identification with Token Permutation (AAAI 2024 · similarity 0.90)
- DeMo: Decoupled Feature-Based Mixture of Experts for Multi-Modal Object Re-Identification (AAAI 2025 · similarity 0.90)
- MDReID: Modality-Decoupled Learning for Any-to-Any Multi-Modal Object Re-Identification (NeurIPS 2025 · similarity 0.89)
- Multi-Modal Object Re-identification via Sparse Mixture-of-Experts (ICML 2025 · similarity 0.89)
- MambaPro: Multi-Modal Object Re-identification with Mamba Aggregation and Synergistic Prompt (AAAI 2025 · similarity 0.88)
- UGG-ReID: Uncertainty-Guided Graph Model for Multi-Modal Object Re-Identification (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.