Research map / Video / Video segmentation and tracking
TF-CLIP: Learning Text-Free CLIP for Video-Based Person Re-identification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TF-CLIP, a one-stage text-free CLIP framework for video-based person re-identification that replaces text features with identity-specific CLIP-Memory, updated online by Sequence-Specific Prompts, plus a Temporal Memory Diffusion module for temporal modeling.
Problem
Transferring CLIP's language-image knowledge to video-based person ReID, where identity labels are integer indices without text descriptions, and temporal information is not modeled by existing CLIP-based ReID methods.
Approach
Replace CLIP's text branch with an identity-specific sequence-feature memory (CLIP-Memory) that is refined per sequence by a cross-attention Sequence-Specific Prompt module, and add Temporal Memory Diffusion that builds frame memory tokens, lets them interact, and diffuses them back to patch tokens, all trained in a single stage.
Key results
- On MARS, TF-CLIP reaches 89.4% mAP and 93.0% Rank-1, surpassing CAVIT by 2.2% in both.
- On LS-VID, TF-CLIP reaches 83.8% mAP and 90.4% Rank-1, versus 82.1% Rank-1 for STMN.
- On iLIDS-VID, the table reports 94.5% Rank-1 and 99.1% Rank-5, while the text states 94.2% Rank-1.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace CLIP-ReID text branch with identity-specific CLIP-Memory (ablation) | video-based person ReID · MARS | mAP: 88.1 -> 88.4 | Helped |
| add SSP to CLIP-Memory (ablation) | Model 2 · MARS | mAP: 88.4 -> 88.8 | Helped |
| add SSP to CLIP-Memory and TMD (ablation) | Model 4 · LS-VID | Rank-1: 88.9 -> 90.4 | Helped |
| add TMD to CLIP-Memory (ablation) | Model 2 · MARS | mAP: 88.4 -> 89.2 | Helped |
| add TMD to CLIP-Memory and SSP (ablation) | Model 3 · LS-VID | Rank-1: +2.5% Rank-1 | Helped |
| use TMD instead of Conv1D temporal fusion (ablation) | Model 2 · MARS | Rank-1: +1.1% | Helped |
| vary SSP depth from N=2 to N=1, N=3, or N=4 (ablation) | Model 3 · MARS | mAP: 88.8; 88.5; 87.4; 88.4 | Mixed |
| remove CLIP-Memory (ablation) | Model 2 · MARS | mAP: 88.4 -> 84.9 | Hurt |
| remove CLIP-Memory (ablation) | Model 4 · MARS | mAP: 89.2 -> 87.3 | Hurt |
| replace TMD with TAP, Conv1D, Transf_cls, or Transf (ablation) | Model 2 temporal fusion · MARS | mAP: 89.2 (TMD); 88.4 (TAP); 88.1 (Conv1D); 87.5 (Transf_cls); 88.2 (Transf) | Mixed |
Surprising findings
- Removing CLIP-Memory drops MARS mAP from 88.4 to 84.9 for the unimodal baseline, suggesting the CLIP text-free memory is essential rather than optional.
Limitations
- Experiments only on three video ReID benchmarks; no evaluation on other ReID settings.
- Reported iLIDS-VID Rank-1 is inconsistent between text (94.2%) and table (94.5%).
Experimental setup
Datasets: MARS LS-VID iLIDS-VID
Benchmarks: MARS LS-VID iLIDS-VID
Metrics: mAP Rank-1 Rank-5
Baselines: STMP M3D GLTR TCLNet MGH GRL BiCnet-TKS CTL STMN PSTA DIL STT
Builds on: CLIP CLIP-ReID CoCoOp ActionCLIP
Closest papers by meaning
- CLIMB-ReID: A Hybrid CLIP-Mamba Framework for Person Re-Identification (AAAI 2025 · similarity 0.90)
- An Empirical Study of CLIP for Text-Based Person Search (AAAI 2024 · similarity 0.88)
- CLIP-ReID: Exploiting Vision-Language Model for Image Re-identification without Concrete Text Labels (AAAI 2023 · similarity 0.88)
- Rethinking Temporal Fusion for Video-Based Person Re-Identification on Semantic and Time Aspect (AAAI 2020 · similarity 0.87)
- Distilling CLIP with Dual Guidance for Learning Discriminative Human Body Shape Representation (CVPR 2024 · similarity 0.86)
- Learning Continual Compatible Representation for Re-indexing Free Lifelong Person Re-identification (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.