Research map / Vision-language and multimodal / Human and embodied modeling
Vocabulary-Guided Gait Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Gait-World, a vocabulary-guided gait recognition paradigm where eight gait-cycle phase words are encoded by frozen VLM/LLM text encoders and used to guide silhouette-based gait networks via the alpha-Gait model (VRM + GFD).
Problem
Gait recognition relies on gait modalities alone and limited databases, making it hard to learn universal gait features and to articulate what a gait concept is; VLMs are insensitive to fine-grained gait silhouettes.
Approach
Use the eight clinically defined gait-cycle phases as a textual Vocabulary Base, encode them with a frozen text encoder (CLIP, LLaMA3-8B, DeepSeek-R1-Distill), and bridge the modality gap with a Vocabulary Relation Mapper (feature transition plus attention) and a Gait Fine-grained Detector (DETR-style query over gait part features) that guide silhouette gait features.
Key results
- alpha-Gait-T reaches 94.8% average Rank-1 on CASIA-B, with 98.9% on NM and 96.8% on BG.
- alpha-Gait-M/L reaches 76.3% Rank-1 and 67.8% mAP on Gait3D, surpassing GaitMoE by 2.6% Rank-1.
- On Gait3D, adding the vocabulary guidance to the Gait Encoder baseline raises Rank-1 from 73.6 to 76.2 and mAP from 65.1 to 67.8.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add vocabulary guidance to the Gait Encoder (ablation) | Gait Encoder · Gait3D | Rank-1: 73.6 to 76.2 (+2.6) | Helped |
| replace text features with random features (ablation) | α-Gait · Gait3D | Rank-1: 76.2 to 71.8 | Hurt |
| replace text features with learnable queries (ablation) | α-Gait · Gait3D | Rank-1: 76.2 to 74.1 | Hurt |
| use CLIP text encoder (ablation) | α-Gait · Gait3D | Rank-1: 76.2 to 75.2 | Hurt |
| use LlaMa text encoder (ablation) | α-Gait · Gait3D | Rank-1: 76.2 to 74.8 | Hurt |
| add α-Gait vocabulary guidance to SkeletonGait (ablation) | SkeletonGait · Gait3D | Rank-1: 37.9 to 40.3 (+2.4) | Helped |
| add α-Gait vocabulary guidance to SkeletonGait++ (ablation) | SkeletonGait++ · Gait3D | Rank-1: 76.0 to 78.1 (+2.1) | Helped |
| expand vocabulary with view-angle, bag, and clothing invariants (ablation) | α-Gait · Gait3D | Rank-1: 76.3 to 76.8 (+0.5) | Helped |
| use α-Gait rather than the Gait Encoder baseline (ablation) | Gait Encoder · CCPG | Mean accuracy: 85.5 to 87.3 | Helped |
| use α-Gait-S rather than DeepGaitV2 | CCPG Gait Evaluation Protocol · CCPG | Mean accuracy: 83.3 to 87.3 | Helped |
| use α-Gait-T rather than the strongest reported baseline | CASIA-B · CASIA-B | Mean Rank-1 accuracy (%): 94.9 to 94.8 | No clear effect |
| use α-Gait-S rather than DeepGaitV2 | SUSTech1K · SUSTech1K | Rank-1: 82.3 to 86.3 | Helped |
Surprising findings
- Random-initialized and learnable-query text features still improve over the baseline gait encoder in some settings, and CLIP and LLaMA3 features perform similarly to DeepSeek-R1-Distill, suggesting that the architecture contributes substantially beyond the specific vocabulary embedding.
Limitations
- Uses only eight gait-cycle vocabulary embeddings, an initial attempt with limited exploitation of vocabulary information.
- No error bars or statistical significance reported.
- Compute resources not reported.
Experimental setup
Datasets: CASIA-B CCPG SUSTech1K Gait3D GREW
Benchmarks: CASIA-B CCPG SUSTech1K Gait3D GREW
Metrics: Rank-1 accuracy Rank-5 accuracy mAP mean accuracy (CCPG)
Baselines: GaitSet GaitPart GaitGL GaitBase DeepGaitV2 DyGait VPNet SkeletonGait GaitMoE CLTD
Builds on: DeepGaitV2 DETR GaitBase CLIP
Closest papers by meaning
- Bridging Gait Recognition and Large Language Models Sequence Modeling (CVPR 2025 · similarity 0.90)
- Learning Visual Prompt for Gait Recognition (CVPR 2024 · similarity 0.88)
- BiggerGait: Unlocking Gait Recognition with Layer-wise Representations from Large Vision Models (NeurIPS 2025 · similarity 0.87)
- Cross-Covariate Gait Recognition: A Benchmark (AAAI 2024 · similarity 0.87)
- RA-GAR: A Richly Annotated Benchmark for Gait Attribute Recognition (AAAI 2025 · similarity 0.86)
- SkeletonGait: Gait Recognition Using Skeleton Maps (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.