Research map / Vision-language and multimodal / Cross-modal learning
Causality-Inspired Invariant Representation Learning for Text-Based Person Retrieval
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes IRLT, a model-agnostic causal-inspired invariant representation learning add-on for text-based person retrieval, using a style intervener (feature-statistic perturbation with InfoNCE) and an ID-wise scene simulator with an IRM-style variance penalty.
Problem
Text-based person retrieval models rely on ERM and spurious statistical correlations (e.g., background, illumination, pose, similar attributes), making retrieval unreliable under environmental changes.
Approach
View TPR from a causal perspective and enforce visual features to be independent of non-causal style (via feature-statistic uncertainty perturbation with contrastive consistency) and sufficient across automatically simulated environments (ID-wise scene splits with an IRM variance penalty), without environment annotations.
Key results
- IRLT improves Rank-1 on all three datasets for each backbone, e.g. Baseline-CLIP on CUHK-PEDES 70.49 to 73.13 and IRRA 73.38 to 74.46.
- IRLT on IRRA reaches 64.72 Rank-1 on ICFG-PEDES and 61.49 on RSTPReid, versus 63.46 and 60.20 without it.
- Cross-dataset generalization (CUHK to ICFG) improves Rank-1 by about 1.1 to 1.8 points over SRCF and IRRA.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add IRLT to IRRA | IRRA · CUHK-PEDES | Rank-1: 73.38 -> 74.46 | Helped |
| Add IRLT to IRRA | IRRA · ICFG-PEDES | Rank-1: 63.46 -> 64.72 | Helped |
| Add IRLT to IRRA | IRRA · RSTPReid | Rank-1: 60.20 -> 61.49 | Helped |
| Add the style intervener (ablation) | SRCF · CUHK-PEDES | Rank-1: 64.04 -> 65.23 | Helped |
| Add the scene simulator (ablation) | SRCF · CUHK-PEDES | Rank-1: 64.04 -> 65.46 | Helped |
| Add the style intervener (ablation) | SRCF · ICFG-PEDES | Rank-1: 57.18 -> 58.15 | Helped |
| Add the scene simulator (ablation) | SRCF · ICFG-PEDES | Rank-1: 57.18 -> 57.91 | Helped |
| Add the style intervener (ablation) | IRRA · ICFG-PEDES | Rank-1: 63.46 -> 64.26 | Helped |
| Add the scene simulator (ablation) | IRRA · ICFG-PEDES | Rank-1: 63.46 -> 63.81 | Helped |
| Add the style intervener (ablation) | IRRA · RSTPReid | Rank-1: 60.20 -> 61.13 | Helped |
| Add the scene simulator (ablation) | IRRA · RSTPReid | Rank-1: 60.20 -> 60.74 | Helped |
Surprising findings
- Gains are smaller on CLIP-based IRRA than on non-CLIP SRCF and SSAN, which the authors attribute to CLIP's pretrained cross-modal knowledge.
Limitations
- Causal factors are not explicitly disentangled from images; relies on approximations of style and environments.
- Gains on the strongest CLIP-based backbone are smaller than on weaker backbones.
Experimental setup
Datasets: CUHK-PEDES ICFG-PEDES RSTPReid
Benchmarks: CUHK-PEDES ICFG-PEDES RSTPReid
Metrics: Rank-1 Rank-5 Rank-10
Baselines: SSAN SRCF IRRA LGUR CFine IVT TIPCB DSSL
Builds on: DSU AdaIN Invariant Risk Minimization REx InfoNCE
Closest papers by meaning
- Adaptive Uncertainty-Based Learning for Text-Based Person Retrieval (AAAI 2024 · similarity 0.87)
- Empowering Visible-Infrared Person Re-Identification with Large Foundation Models (NeurIPS 2024 · similarity 0.86)
- Multi-Modal Disordered Representation Learning Network for Description-Based Person Search (AAAI 2024 · similarity 0.86)
- An Empirical Study of CLIP for Text-Based Person Search (AAAI 2024 · similarity 0.85)
- Identification of Necessary Semantic Undertakers in the Causal View for Image-Text Matching (AAAI 2024 · similarity 0.85)
- Automatic Synthetic Data and Fine-grained Adaptive Feature Alignment for Composed Person Retrieval (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.