Research map / Image generation / Fast diffusion sampling
Diffusion-based Synthetic Data Generation for Visible-Infrared Person Re-Identification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DiVE, which generates identity-preserving RGB-IR image pairs by decoupling identity (textual inversion) and modality (LoRA fine-tuned Stable Diffusion 1.5) to expand VI-ReID training data.
Problem
Visible-infrared person re-identification is limited by the scarcity and high annotation cost of cross-modal RGB-IR training data, and existing generative methods do not preserve identity when translating RGB to IR.
Approach
Represent identity with per-ID textual embeddings learned by textual inversion from same-ID images, and modality (multi-view IR/RGB) with fixed placeholder tokens and a shared LoRA on Stable Diffusion, then recombine them to synthesize identity-consistent IR images from external RGB ReID datasets such as Market-1501.
Key results
- DEEN trained with DiVE-synthetic data improves SYSU-MM01 all-search Rank-1 from 74.70% to 79.07% and mAP from 71.80% to 74.96%.
- CAJ with DiVE data improves LLCM VIS-to-IR mAP from 56.40% to 59.40%, with IR-to-VIS mAP gaining about 9 points (47.71 to 56.80).
- DiVE achieves FID 47.92 for synthetic infrared images versus 125.98 for the best baseline (CycleGAN-Turbo).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add DiVE-generated synthetic data | DEEN · SYSU-MM01 all-search | R-1: 74.70 -> 79.07 | Helped |
| Add DiVE-generated synthetic data | DEEN · SYSU-MM01 all-search | mAP: 71.80 -> 74.96 | Helped |
| Add DiVE-generated synthetic data | DEEN · LLCM VIS to IR | mAP: 62.07 -> 65.90 | Helped |
| Add DiVE-generated synthetic data | DEEN · LLCM IR to VIS | mAP: 55.52 -> 59.43 | Helped |
| Use RGB+IR data across multiple views rather than a single view (ablation) | DiVE-expanded DEEN training · SYSU-MM01 | R1: 76.89 -> 79.07 | Helped |
| Use RGB+IR data across multiple views rather than a single view (ablation) | DiVE-expanded DEEN training · SYSU-MM01 | mAP: 72.98 -> 74.96 | Helped |
| Add RGB modality to IR-only data (ablation) | DiVE-expanded DEEN training · SYSU-MM01 | R1: 75.59 -> 76.89 | Helped |
| Add RGB modality to IR-only data (ablation) | DiVE-expanded DEEN training · SYSU-MM01 | mAP: 71.89 -> 72.98 | Helped |
| Use CUHK-NP as the visible source dataset (ablation) | DiVE-expanded DEEN training · SYSU-MM01 | R1: 74.70 -> 77.50 | Helped |
| Use Market-1501 as the visible source dataset (ablation) | DiVE-expanded DEEN training · SYSU-MM01 | R1: 74.70 -> 79.07 | Helped |
| Increase augmented identity count beyond the reported peak of 1011 (ablation) | DiVE-expanded DEEN training · SYSU-MM01 | Rank-1 accuracy | Hurt |
| Use DiVE rather than CycleGAN, AlignGAN, CycleGAN-Turbo, or VI-Diff for IR generation | Synthetic IR generation · SYSU-MM01 | FID: 47.9249 versus 175.1905, 128.1884, 125.9849, and 144.8423 | Helped |
Surprising findings
- Adding more augmented identities beyond a point harms ReID performance, attributed to long-tailed ID distributions making ID feature extraction from few images hard.
Limitations
- Synthetic IR images sometimes retain identity-specific details from IR training samples, so identity and modality are not fully decoupled (identity leakage).
- Pose controllability is limited; poses vary randomly during generation.
- Performance declines when too many augmented IDs are added, due to long-tailed per-ID image counts.
Experimental setup
Datasets: SYSU-MM01 LLCM Market-1501 CUHK03-NP
Benchmarks: SYSU-MM01 LLCM Market-1501 CUHK03-NP
Metrics: Rank-1 mAP FID
Baselines: CycleGAN AlignGAN VI-Diff CycleGAN-Turbo AGW CAJ DART DEEN
Builds on: Stable Diffusion Textual Inversion DreamBooth LoRA Diff-Mix
Closest papers by meaning
- Learning Commonality, Divergence and Variety for Unsupervised Visible-Infrared Person Re-identification (NeurIPS 2024 · similarity 0.88)
- TokenMatcher: Diverse Tokens Matching for Unsupervised Visible-Infrared Person Re-Identification (AAAI 2025 · similarity 0.87)
- From Laboratory to Real World: A New Benchmark Towards Privacy-Preserved Visible-Infrared Person Re-Identification (CVPR 2025 · similarity 0.87)
- From Poses to Identity: Training-Free Person Re-Identification via Feature Centralization (CVPR 2025 · similarity 0.87)
- Empowering Visible-Infrared Person Re-Identification with Large Foundation Models (NeurIPS 2024 · similarity 0.87)
- Instruct-ReID: A Multi-purpose Person Re-identification Task with Instructions (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.