Research map / Detection and segmentation / CNN features
TOP-ReID: Multi-Spectral Object Re-identification with Token Permutation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TOP-ReID, a multi-stream ViT framework for RGB/NIR/TIR object re-identification that uses cyclic token permutation (TPM) and token-level complementary reconstruction (CRM) to align spectra and handle missing spectra.
Problem
Multi-spectral object re-identification suffers from large distribution gaps among RGB, NIR and TIR images, existing Transformer ReID methods use only class tokens, and spectra are often missing at test time.
Approach
A Token Permutation Module cyclically exchanges each spectrum's class token with the patch tokens of other spectra across three consecutive permutations, so class tokens absorb local detail from other spectra; a Complementary Reconstruction Module adds dense MSE constraints that reconstruct missing spectrum tokens, reducing the inter-spectrum gap.
Key results
- On RGBNT201, TOP-ReID reaches 72.3 mAP and 76.6 Rank-1 versus IEEE's 47.5 mAP and 44.4 Rank-1 (multi-spectral baseline).
- On RGBNT100, TOP-ReID reaches 81.2 mAP, about 4.0 points above CCNet (77.2).
- On MSVR310, TOP-ReID achieves 35.9 mAP, below CCNet's 36.4, so it is competitive rather than best.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TOP-ReID instead of IEEE | RGBNT201 multi-spectral ReID · RGBNT201 | mAP: 47.5 -> 72.3 | Helped |
| Use TOP-ReID instead of PHT | RGBNT100 multi-spectral ReID · RGBNT100 | mAP: 79.9 -> 81.2 | Helped |
| Use TOP-ReID instead of CCNet | MSVR310 multi-spectral ReID · MSVR310 | R-1: 55.2 -> 44.6 | Hurt |
| Add TPM to the AL baseline (ablation) | TOP-ReID on RGBNT201 · RGBNT201 | mAP: 62.9 -> 67.8 | Helped |
| Add CRM to the AL + TPM variant (ablation) | TOP-ReID on RGBNT201 · RGBNT201 | mAP: 67.8 -> 72.3 | Helped |
| Use AL supervision after feature concatenation instead of BL before concatenation (ablation) | TOP-ReID baseline on RGBNT201 · RGBNT201 | mAP: 55.9 -> 62.9 | Helped |
| Add TPM to the BL baseline (ablation) | TOP-ReID on RGBNT201 · RGBNT201 | mAP: 55.9 -> 59.4 | Helped |
| Add CRM to the BL + TPM variant (ablation) | TOP-ReID on RGBNT201 · RGBNT201 | mAP: 59.4 -> 64.6 | Helped |
| Add the second TPM permutation (ablation) | TPM on RGBNT201 · RGBNT201 | mAP: 65.1 -> 66.5 | Helped |
| Add the third TPM permutation (ablation) | TPM on RGBNT201 · RGBNT201 | mAP: 66.5 -> 67.8 | Helped |
| Replace CRM's MSE loss with MAE (ablation) | CRM on RGBNT201 · RGBNT201 | mAP: 72.3 -> 68.9 | Hurt |
| Replace CRM's MSE loss with RMSE (ablation) | CRM on RGBNT201 · RGBNT201 | mAP: 72.3 -> 69.8 | Hurt |
Surprising findings
- On MSVR310 the method does not beat CCNet in mAP or Rank-1, a case where the proposed approach is only competitive.
Limitations
- Missing-spectrum handling relies on reconstruction quality, which is weaker when two spectra are missing (e.g., M(N+T) mAP 34.1 on RGBNT201).
- Performance on the small-scale MSVR310 dataset does not surpass CCNet.
- Backbone results show stronger gains with ViT-B/16 than with other backbones.
Experimental setup
Datasets: RGBNT201 RGBNT100 MSVR310 ImageNet
Benchmarks: RGBNT201 RGBNT100 MSVR310
Metrics: mAP CMC Rank-1 Rank-5 Rank-10
Baselines: HACNN MUDeep OSNet MLFN CAL PCB HAMNet PFNet DENet IEEE CCNet TransReID
Builds on: TransReID vision transformer label smoothing cross-entropy triplet loss
Closest papers by meaning
- Magic Tokens: Select Diverse Tokens for Multi-modal Object Re-Identification (CVPR 2024 · similarity 0.90)
- UGG-ReID: Uncertainty-Guided Graph Model for Multi-Modal Object Re-Identification (NeurIPS 2025 · similarity 0.87)
- DeMo: Decoupled Feature-Based Mixture of Experts for Multi-Modal Object Re-Identification (AAAI 2025 · similarity 0.87)
- Catalyst for Clustering-Based Unsupervised Object Re-identification: Feature Calibration (AAAI 2024 · similarity 0.86)
- MambaPro: Multi-Modal Object Re-identification with Mamba Aggregation and Synergistic Prompt (AAAI 2025 · similarity 0.85)
- High-Order Structure Based Middle-Feature Learning for Visible-Infrared Person Re-identification (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.