Research map / Vision-language and multimodal / Human and embodied modeling
Instruct-ReID: A Multi-purpose Person Re-identification Task with Instructions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes instruct-ReID, a unified person re-identification task driven by image or language instructions, with the OmniReID benchmark (12 datasets), an editing-transformer model (IRM), and an adaptive triplet loss.
Problem
Person re-identification is studied as separate tasks (traditional, clothes-changing, clothes-template, visible-infrared, text-to-image), so one deployable model cannot handle all retrieval scenarios.
Approach
Cast all ReID variants as instruction-conditioned retrieval; an editing transformer injects instruction features into query image tokens via gated cross-attention, trained with an adaptive triplet loss whose margin depends on instruction similarity.
Key results
- Multi-task IRM reaches 52.3% mAP on PRCC and 52.0% on LTCC for clothes-changing ReID, and improves COCAS+ Real2 language-instructed ReID by +24.9% mAP over the baseline.
- IRM (MTL) achieves 66.5% mAP on CUHK-PEDES, +2.6% over RaSa (63.9%).
- IRM (MTL) achieves 67.2% mAP on LLCM IR-to-VIS, a new state of the art; on Market1501, MSMT17 and CUHK03 it gains +0.5%, +0.6% and +7.7% mAP over single-task training.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train jointly on multiple ReID tasks and datasets instead of one task | IRM · COCAS+ Real2 CTCC-ReID | mAP: 32.2 -> 41.7 | Helped |
| Train jointly on multiple ReID tasks and datasets instead of one task | IRM · COCAS+ Real2 LI-ReID | mAP: 30.7 -> 39.8 | Helped |
| Train jointly on multiple ReID tasks and datasets instead of one task | IRM · VC-Clothes | mAP: 80.1 -> 78.9 | Hurt |
| Train jointly on multiple ReID tasks and datasets instead of one task | IRM · Market1501 | mAP: 92.3 -> 93.5 | Helped |
| Remove the editing transformer (ablation) | IRM (STL) · COCAS+ Real2 CTCC-ReID | mAP: 32.2 -> 32.6 | No clear effect |
| Remove the editing transformer (ablation) | IRM (STL) · COCAS+ Real2 LI-ReID | mAP: 30.7 -> 30.5 | No clear effect |
| Replace adaptive triplet loss with traditional triplet loss (ablation) | IRM (STL) · COCAS+ Real2 CTCC-ReID | mAP: 31.5 -> 32.2 | Hurt |
| Replace adaptive triplet loss with traditional triplet loss (ablation) | IRM (STL) · COCAS+ Real2 LI-ReID | mAP: 30.2 -> 30.7 | Hurt |
| Replace adaptive triplet loss with traditional triplet loss (ablation) | IRM (MTL) · COCAS+ Real2 CTCC-ReID | mAP: 40.7 -> 41.7 | Hurt |
| Replace adaptive triplet loss with traditional triplet loss (ablation) | IRM (MTL) · COCAS+ Real2 LI-ReID | mAP: 39.2 -> 39.8 | Helped |
| Replace adaptive triplet loss with traditional triplet loss (ablation) | IRM (MTL) · LLCM VIS-to-IR | mAP: 68.4 -> 67.5 | Hurt |
| Replace adaptive triplet loss with traditional triplet loss (ablation) | IRM (MTL) · PRCC | mAP: 52.4 -> 52.3 | No clear effect |
Surprising findings
- The editing transformer design gives a -0.6% mAP drop in the multi-task setting compared with a plain ViT-Base, so the instruction-fusion module does not help there.
- Multi-task learning slightly lowers VC-Clothes mAP relative to single-task training, which the authors attribute to a synthetic-to-real domain gap.
Limitations
- Instructions are a fixed set of GPT-4-generated sentences, and the authors leave better instruction design to future work.
- Multi-task learning shows a performance drop on the synthetic VC-Clothes dataset, which the authors attribute to domain gap.
- Results are reported at 256x128 resolution; higher-resolution comparisons are only partly covered.
Experimental setup
Datasets: Market1501 MSMT17 CUHK03 PRCC VC-Clothes LTCC LLCM CUHK-PEDES SYNTH-PEDES COCAS+ Real1 COCAS+ Real2
Benchmarks: Market1501 MSMT17 CUHK03 PRCC VC-Clothes LTCC LLCM CUHK-PEDES COCAS+ Real2
Metrics: mAP Top-1 CMC
Baselines: TransReID CAL BC-Net DualBCT-Net RaSa PSLD SAF DEEN PASS SAN HumanBench
Builds on: Instruct-BLIP TransReID ALBEF triplet loss
Closest papers by meaning
- Multi-Prompts Learning with Cross-Modal Alignment for Attribute-Based Person Re-identification (AAAI 2024 · similarity 0.88)
- LLaVA-ReID: Selective Multi-image Questioner for Interactive Person Re-Identification (ICML 2025 · similarity 0.88)
- Distilling CLIP with Dual Guidance for Learning Discriminative Human Body Shape Representation (CVPR 2024 · similarity 0.88)
- Exploring the Camera Bias of Person Re-identification (ICLR 2025 · similarity 0.88)
- ReID5o: Achieving Omni Multi-modal Person Re-identification in a Single Model (NeurIPS 2025 · similarity 0.87)
- Person De-reidentification: A Variation-guided Identity Shift Modeling (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.