Research map / Vision-language and multimodal / Human and embodied modeling
UFineBench: Towards Text-based Person Retrieval with Ultra-fine Granularity
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces UFineBench, a text-based person retrieval benchmark with ultra-fine-grained captions (UFine6926), a cross-setting evaluation set (UFine3C), a new mSD metric, and the CFAM CLIP-based model.
Problem
Existing text-based person retrieval datasets have coarse-grained, often identity-ambiguous text annotations and fixed evaluation settings, so they do not reflect fine-grained query understanding or generalization to real scenarios; rank-k/mAP metrics also fail to measure continuous similarity diffe
Approach
Collects person images from unrestricted internet videos with two long (avg 80.8-word) manual descriptions per image, builds a cross-domain/granularity/style evaluation set, proposes mSD (similarity-distribution metric), and CFAM, which uses a shared cross-modal granularity decoder with query tokens plus hard-negative matching on top of CLIP.
Key results
- On UFine3C (CLIP-ViT-L/14 setting), CFAM reaches 62.84% rank-1 when trained on UFine6926 (B/16 or L/14 numbers differ across tables), with 46.04% mSD, beating IRRA and PLIP trained on coarse datasets.
- CFAM (L/14) achieves 76.71% rank-1 on CUHK-PEDES, 66.58% rank-1 on ICFG-PEDES and 63.54% rank-1 on RSTPReid, reported as state-of-the-art.
- Models trained on coarse-grained CUHK-PEDES transfer poorly to UFine6926 (PLIP 20.52% rank-1), while training on UFine6926 transfers better to coarse datasets.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train CFAM on UFine6926 rather than CUHK-PEDES | CFAM · UFine3C | R@1: 53.80 -> 62.84 | Helped |
| train CFAM on UFine6926 rather than ICFG-PEDES | CFAM · UFine3C | R@1: 36.79 -> 62.84 | Helped |
| train CFAM on UFine6926 rather than RSTPReid | CFAM · UFine3C | R@1: 29.85 -> 62.84 | Helped |
| add global alignment loss (ablation) | CFAM · CUHK-PEDES | R@1: 68.45 -> 70.42 | Helped |
| add local alignment loss (ablation) | CFAM with global alignment · CUHK-PEDES | R@1: 70.42 -> 71.69 | Helped |
| add hard negative matching loss (ablation) | CFAM with global and local alignment · CUHK-PEDES | R@1: 71.69 -> 72.42 | Helped |
| add Lvap (ablation) | CFAM with global alignment, local alignment, and hard negative matching · CUHK-PEDES | R@1: 72.42 -> 73.67 | Helped |
Surprising findings
- Training on the smaller UFine6926 (about 4.9k identities) generalizes better to UFine3C and to coarse datasets than training on larger coarse-grained datasets.
- Coarse-grained-trained models transfer poorly to the ultra-fine-grained dataset, while the reverse direction transfers better.
Limitations
- UFine6926 is relatively small in identities (6,926) and the cross-setting evaluation set relies partly on LLM-rewritten and manually revised text.
- mSD is defined with a hyperparameter k set to 1 by default and is a new metric without broad adoption or sensitivity analysis in the text.
Experimental setup
Datasets: UFine6926 CUHK-PEDES ICFG-PEDES RSTPReid UFine3C
Benchmarks: UFine3C UFine6926 CUHK-PEDES ICFG-PEDES RSTPReid
Metrics: rank-1 rank-5 rank-10 mAP mSD
Baselines: IRRA PLIP NAFS LGUR SSAN CFine IVT LBUL
Builds on: CLIP IRRA SDM loss FairMOT PLIP
Closest papers by meaning
- Chat-based Person Retrieval via Dialogue-Refined Cross-Modal Alignment (CVPR 2025 · similarity 0.86)
- Automatic Synthetic Data and Fine-grained Adaptive Feature Alignment for Composed Person Retrieval (NeurIPS 2025 · similarity 0.85)
- Adaptive Uncertainty-Based Learning for Text-Based Person Retrieval (AAAI 2024 · similarity 0.85)
- Exploring the Potential of Large Vision-Language Models for Unsupervised Text-Based Person Retrieval (AAAI 2025 · similarity 0.85)
- DM-Adapter: Domain-Aware Mixture-of-Adapters for Text-Based Person Retrieval (AAAI 2025 · similarity 0.84)
- Diverse Person: Customize Your Own Dataset for Text-Based Person Search (AAAI 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.