Research map / Vision-language and multimodal / Human and embodied modeling
Automatic Synthetic Data and Fine-grained Adaptive Feature Alignment for Composed Person Retrieval
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Composed Person Retrieval (CPR), builds a fully synthetic 1.15M-triplet SynCPR dataset via LLM quadruple generation, LoRA-fine-tuned Flux image synthesis and MLLM filtering, and proposes the FAFA BLIP-2-based framework evaluated on a manually annotated ITCPR test set.
Problem
Person retrieval typically uses either image-only or text-only queries, failing to exploit combined visual and textual queries; composed person retrieval lacks annotated training data because collecting same-identity image pairs with relative captions is costly and raises privacy issues.
Approach
Decompose composed triplet synthesis into LLM-generated textual quadruples, identity-consistent two-in-one image generation with a LoRA-fine-tuned Flux model, and MLLM-based filtering; then train FAFA, a BLIP-2 Q-Former model that dynamically aligns the query with top-k fine-grained target features, adds feature diversity loss and masked feature reconstruction.
Key results
- FAFA achieves 46.54 Rank-1 and 55.60 mAP on the ITCPR test set, versus 42.27 Rank-1 for the best CIR baseline (SPRC trained on SynCPR).
- Training on SynCPR lifts SPRC from 25.07 to 42.27 Rank-1 on ITCPR, showing the value of the synthetic dataset.
- FAFA reaches 54.48 Rank-1 on CIRR, a 2.51-point gain over SPRC's 51.96.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SynCPR instead of CIRR training data | CaLa · ITCPR | Rank-1: 24.02 -> 39.33 | Helped |
| Use SynCPR instead of CIRR training data | SPRC · ITCPR | Rank-1: 25.07 -> 42.27 | Helped |
| Replace ITC with fine-grained dynamic alignment (FDA) (ablation) | FAFA · ITCPR | Rank-1: 41.33 -> 45.04 | Helped |
| Add feature diversity supervision (FD) (ablation) | FAFA with FDA · ITCPR | Rank-1: 45.04 -> 46.05 | Helped |
| Add masked feature reasoning (MFR) (ablation) | FAFA with FDA · ITCPR | Rank-1: 45.04 -> 45.78 | Helped |
| Add MFR to FDA and FD (ablation) | FAFA · ITCPR | Rank-1: 46.05 -> 46.54 | Helped |
| Add FD to FDA and MFR (ablation) | FAFA · ITCPR | Rank-1: 45.78 -> 46.54 | Helped |
| Use FAFA instead of SPRC | CIRR retrieval · CIRR | Rank-1: 51.96 -> 54.48 | Helped |
Surprising findings
- Supervised CIR methods trained on original CIR datasets underperform some zero-shot CIR methods on the CPR task.
Limitations
- Future work is needed for multi-image and multi-text composed queries, which the paper does not cover.
- Evaluation is limited to the ITCPR test set of 2,225 triplets built from clothes-changing datasets.
- Filtering relies on an MLLM judge (Qwen2.5-VL-32B) and a strict threshold that discards about 59% of generated data.
Experimental setup
Datasets: SynCPR CUHK-PEDES CIRR Celeb-reID PRCC LAST
Benchmarks: ITCPR CIRR
Metrics: Rank-1 Rank-5 Rank-10 mAP Rank-K Rank_s-K
Baselines: TransReID SOLIDER CLIP-ReID CAL FIRe2 RaSa IRRA RDE Pic2Word CoVR-BLIP LinCIR CaLa
Builds on: BLIP-2 Flux.1 LoRA SPRC CaLa
Closest papers by meaning
- Adaptive Uncertainty-Based Learning for Text-Based Person Retrieval (AAAI 2024 · similarity 0.88)
- Exploring the Potential of Large Vision-Language Models for Unsupervised Text-Based Person Retrieval (AAAI 2025 · similarity 0.86)
- Chat-based Person Retrieval via Dialogue-Refined Cross-Modal Alignment (CVPR 2025 · similarity 0.86)
- DM-Adapter: Domain-Aware Mixture-of-Adapters for Text-Based Person Retrieval (AAAI 2025 · similarity 0.86)
- An Empirical Study of CLIP for Text-Based Person Search (AAAI 2024 · similarity 0.86)
- Human-centered Interactive Learning via MLLMs for Text-to-Image Person Re-identification (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.