Research map / Label-efficient and robust learning / Test-time adaptation
Heterogeneous Test-Time Training for Multi-Modal Person Re-identification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Heterogeneous Test-time Training (HTT) for multi-modal person ReID, combining a cross-identity inter-modal margin loss with test-time training on unlabeled test data using self-supervised 3M and CIM losses.
Problem
Multi-modal person re-identification under challenging lighting conditions, where existing methods focus on modality fusion and ignore inter-modal relationships and the unlabeled test-domain data at inference time.
Approach
A cross-identity inter-modal margin (CIM) loss constrains inter-modal distances between anchor and negative samples from different identities, and a multi-modal test-time training strategy fine-tunes the model on unlabeled test data using self-supervised 3M and CIM losses, with pseudo-labels from the classifier deciding which pairs are negatives.
Key results
- On RGBNT201, full HTT reaches 71.1% mAP and 73.4% Rank-1, versus 46.4% mAP for IEEE.
- On RGBN300, HTT reaches 77.1% mAP, exceeding GAFNet's 72.7%.
- Adding CIM loss to IEEE raises RGBNT201 mAP from 46.5% to 49.3%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add MTT to HTT (ablation) | HTT · RGBNT201 | mAP: 69.0 -> 71.1 | Helped |
| Add MTT to HTT (ablation) | HTT · RGBNT201 | Rank-1: 70.0 -> 73.4 | Helped |
| Add MTT to HTT (ablation) | HTT · Market1501-MM | mAP: 65.9 -> 67.2 | Helped |
| Add MTT to HTT (ablation) | HTT · Market1501-MM | Rank-1: 80.3 -> 81.5 | Helped |
| Add CIM loss to the baseline (ablation) | Baseline · RGBNT201 | mAP: 67.5 -> 69.0 | Helped |
| Add CIM loss to the baseline (ablation) | Baseline · RGBNT201 | Rank-1: 69.4 -> 70.3 | Helped |
| Add CIM loss and MTT (ablation) | Baseline · RGBNT201 | mAP: 67.5 -> 71.1 | Helped |
| Add CIM loss and MTT (ablation) | Baseline · RGBNT201 | Rank-1: 69.4 -> 73.4 | Helped |
| Add CIM loss (ablation) | OSNet · RGBNT201 | mAP: 23.4 -> 24.8 | Helped |
| Add CIM loss (ablation) | OSNet · RGBNT201 | Rank-1: 23.7 -> 23.9 | Helped |
| Add CIM loss (ablation) | IEEE · RGBNT201 | mAP: 46.5 -> 49.3 | Helped |
| Add CIM loss (ablation) | IEEE · RGBNT201 | Rank-1: 47.8 -> 49.2 | Helped |
Limitations
- Test-time training requires pseudo-labels from the classifier, so CIM pair selection depends on their quality.
- Requires batch size of at least 2 and fine-tuning on the test set before inference.
- Authors note that improving the quality of sampled test data is future work.
Experimental setup
Datasets: RGBNT201 Market1501-MM RGBN300 RGBNT100
Benchmarks: RGBNT201 Market1501-MM RGBN300 RGBNT100
Metrics: mAP CMC Rank-1 Rank-5 Rank-10
Baselines: MLFN HACNN OSNet HAMNet PFNet IEEE PCB MGN ABD GAFNet
Builds on: test-time training (Sun et al. 2020) IEEE triplet loss vision transformer
Closest papers by meaning
- ReID5o: Achieving Omni Multi-modal Person Re-identification in a Single Model (NeurIPS 2025 · similarity 0.89)
- Cross-Modality Perturbation Synergy Attack for Person Re-identification (NeurIPS 2024 · similarity 0.87)
- Empowering Visible-Infrared Person Re-Identification with Large Foundation Models (NeurIPS 2024 · similarity 0.87)
- Multi-Prompts Learning with Cross-Modal Alignment for Attribute-Based Person Re-identification (AAAI 2024 · similarity 0.86)
- Implicit Discriminative Knowledge Learning for Visible-Infrared Person Re-Identification (CVPR 2024 · similarity 0.86)
- Instruct-ReID: A Multi-purpose Person Re-identification Task with Instructions (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.