Research map / Video / Video segmentation and tracking
Temporal Correlation Vision Transformer for Video Person Re-Identification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TCViT, a ViT-based two-stage video person Re-ID model with a parameter-free Temporal Correlation Attention (KCF-based patch alignment plus temporal-correlation reweighting) and a Learnable Temporal Aggregation module that weights frames by classification scores.
Problem
Video person Re-ID suffers from occlusion in dense crowds and frame misalignment from detector bounding boxes, and standard temporal pooling treats all frames equally even though some are occluded.
Approach
Use relative state across frames: align patch features with a kernelized correlation filter, weight patches by cosine similarity to the temporal average (static parts likely the target), and learn frame weights for aggregation guided by a classification-score MSE loss.
Key results
- On LS-VID, TCViT reaches 83.1% mAP and 90.1% rank-1, beating the previous best CAViT (79.2% mAP / 89.2% rank-1 as listed).
- On MARS, TCViT achieves 87.6% mAP and 91.7% rank-1, outperforming all listed methods.
- On VVeRI-901, TCViT (8-frame) reaches 69.0% mAP / 63.9% rank-1, improving over CAViT by 3.4 mAP and 3.9 rank-1.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add TCA and LTA (ablation) | ViT-base video Re-ID baseline · LS-VID | mAP: 80.5 -> 83.1 | Helped |
| add filter-based patch alignment to TCA (ablation) | TCViT · LS-VID | mAP: 81.0 -> 81.4 | Helped |
| add TCA (ablation) | ViT-base video Re-ID baseline · LS-VID | mAP: 80.5 -> 81.4 | Helped |
| add LTA without MSE loss guidance (ablation) | ViT-base video Re-ID baseline · LS-VID | mAP: 80.5 -> 81.0 | Helped |
| add MSE loss guidance to LTA (ablation) | TCViT · LS-VID | mAP: 81.0 -> 82.0 | Helped |
| place TCA after layers 3 and 5 instead of layers 7 and 9 (ablation) | TCViT · LS-VID | mAP: 83.1 -> 76.6 | Hurt |
| place TCA after layers 5 and 7 instead of layers 7 and 9 (ablation) | TCViT · LS-VID | mAP: 83.1 -> 81.7 | Hurt |
| place TCA after layers 9 and 11 instead of layers 7 and 9 (ablation) | TCViT · LS-VID | mAP: 83.1 -> 82.7 | Hurt |
| add a third TCA module after layer 11 (ablation) | TCViT · LS-VID | mAP: 83.1 -> 83.0 | No clear effect |
| place TCA after layers 7 and 8 instead of layers 7 and 9 (ablation) | TCViT · LS-VID | mAP: 83.1 -> 82.8 | Hurt |
| use one TCA module after layer 7 instead of two (ablation) | TCViT · LS-VID | mAP: 83.1 -> 82.6 | Hurt |
| set LTA reduction rate r=2 instead of r=8 (ablation) | TCViT · LS-VID | mAP: 83.1 -> 82.1 | Hurt |
Surprising findings
- Applying TCA in shallow layers (Z3+Z5) substantially hurts performance (76.6 mAP vs 83.1), since weakening non-target patches disrupts patch relations learned in shallow layers.
Limitations
- Occlusion-handling gains are modest (about 1 point rank-1 over baseline on LS-VID component ablation).
- Filter-based alignment initialized from the first frame can distort body structure when the first detection is inaccurate.
- Inference time rises to 1.44x baseline due to TCA alignment.
Experimental setup
Datasets: MARS LS-VID iLiDS-VID VVeRI-901
Benchmarks: LS-VID MARS iLiDS-VID VVeRI-901
Metrics: mAP rank-1 rank-5 inference time
Baselines: GLTR VRSTC M3D AP3D TCLNet AFA MGH MG-RAFA BiCnet-TKS GRL STRF STMN
Builds on: TransReID CAViT KCF (kernelized correlation filters) Hou et al. VRSTC temporal average pooling triplet loss
Closest papers by meaning
- Relation-Guided Spatial Attention and Temporal Refinement for Video-Based Person Re-Identification (AAAI 2020 · similarity 0.88)
- Watching You: Global-Guided Reciprocal Learning for Video-Based Person Re-Identification (CVPR 2021 · similarity 0.87)
- Spatial and Temporal Mutual Promotion for Video-Based Person Re-Identification (AAAI 2019 · similarity 0.86)
- Rethinking Temporal Fusion for Video-Based Person Re-Identification on Semantic and Time Aspect (AAAI 2020 · similarity 0.86)
- Learning Modal-Invariant and Temporal-Memory for Video-Based Visible-Infrared Person Re-Identification (CVPR 2022 · similarity 0.86)
- CDE-Learning: Camera Deviation Elimination Learning for Unsupervised Person Re-identification (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.