Research map / Vision-language and multimodal / Cross-modal learning
Unifying Multi-Modal Uncertainty Modeling and Semantic Alignment for Text-to-Image Person Re-identification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Models image and text embeddings as Gaussian distributions with multi-granularity (batch- and identity-level) uncertainty, adds a bi-directional cross-modal circle loss and a cross-modal global semantic recovery task on a CLIP-based TI-ReID model.
Problem
Text-to-image person re-identification relies on deterministic image/text embeddings that ignore the inherent uncertainty of pedestrian images and their textual descriptions, limiting cross-modal semantic alignment.
Approach
Represent global image and text features as Gaussian distributions whose variance combines batch-level and identity-level (memory-bank) feature variances, use the sampled features as augmentation, and align them with a cross-modal circle loss plus a masked global token reconstruction task (cm-GSR) on top of CLIP.
Key results
- On RSTPReid, the method reaches 63.40% Rank-1 and 49.28% mAP, about +3.15% Rank-1 over IRRA.
- On CUHK-PEDES, the method reaches 74.25% Rank-1 and 66.15% mAP, surpassing IRRA (73.38%) by +0.87% Rank-1.
- On ICFG-PEDES, the method reaches 65.62% Rank-1, outperforming IRRA by +2.16% Rank-1.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the full method instead of the CLIP-ViT-B/16 baseline | TI-ReID model · CUHK-PEDES | Rank-1 accuracy: 72.98 -> 74.25 | Helped |
| Use the full method instead of the CLIP-ViT-B/16 baseline | TI-ReID model · RSTPReid | Rank-1 accuracy: 59.80 -> 63.40 | Helped |
| Use the full method instead of the CLIP-ViT-B/16 baseline | TI-ReID model · ICFG-PEDES | Rank-1 accuracy: 63.34 -> 65.62 | Helped |
| Add MUM (ablation) | CLIP-ViT-B/16 baseline · RSTPReid | Rank-1 accuracy: 59.80 -> 62.35 | Helped |
| Add cm-Circle loss (ablation) | CLIP-ViT-B/16 baseline · RSTPReid | Rank-1 accuracy: 59.80 -> 61.50 | Helped |
| Add cm-GSR (ablation) | CLIP-ViT-B/16 baseline · RSTPReid | Rank-1 accuracy: 59.80 -> 61.25 | Helped |
| Add cm-Circle to MUM (ablation) | MUM variant · RSTPReid | Rank-1 accuracy: 62.35 -> 62.45 | Helped |
| Add cm-GSR to MUM (ablation) | MUM variant · RSTPReid | Rank-1 accuracy: 62.35 -> 62.80 | Helped |
| Add cm-GSR to MUM + cm-Circle (ablation) | MUM + cm-Circle variant · RSTPReid | Rank-1 accuracy: 62.45 -> 63.40 | Helped |
| Use batch-level uncertainty modeling (ablation) | CLIP-ViT-B/16 baseline · RSTPReid | Rank-1 accuracy: 59.80 -> 61.20 | Helped |
| Use identity-level uncertainty modeling (ablation) | CLIP-ViT-B/16 baseline · RSTPReid | Rank-1 accuracy: 59.80 -> 61.65 | Helped |
| Combine batch-level and identity-level uncertainty as multi-granularity MUM (ablation) | CLIP-ViT-B/16 baseline · RSTPReid | Rank-1 accuracy: 59.80 -> 62.35 | Helped |
Limitations
- Gains over the strongest prior method (IRRA) on CUHK-PEDES are small (+0.87% Rank-1).
- Uncertainty module relies on memory banks of size 65536 and hyperparameters (omega, s, gamma, m, lambdas) tuned per dataset.
Experimental setup
Datasets: CUHK-PEDES RSTPReid ICFG-PEDES
Benchmarks: CUHK-PEDES RSTPReid ICFG-PEDES
Metrics: Rank-1 Rank-5 Rank-10 mAP
Baselines: IRRA CFine MCM IVT TIPCB TextReID AXM-Net LGUR
Builds on: CLIP IRRA circle loss similarity distribution matching loss probabilistic cross-modal embedding
Closest papers by meaning
- Adaptive Uncertainty-Based Learning for Text-Based Person Retrieval (AAAI 2024 · similarity 0.89)
- Noisy-Correspondence Learning for Text-to-Image Person Re-identification (CVPR 2024 · similarity 0.89)
- Modeling Thousands of Human Annotators for Generalizable Text-to-Image Person Re-identification (CVPR 2025 · similarity 0.89)
- Text-Based Occluded Person Re-identification via Multi-Granularity Contrastive Consistency Learning (AAAI 2024 · similarity 0.88)
- Empowering Visible-Infrared Person Re-Identification with Large Foundation Models (NeurIPS 2024 · similarity 0.88)
- Multi-Prompts Learning with Cross-Modal Alignment for Attribute-Based Person Re-identification (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.