Research map / Large language models / Language, safety and interpretability
Bridging Gait Recognition and Large Language Models Sequence Modeling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GaitLLM, which inserts a frozen pretrained LLM (Qwen-0.5B) between a frozen gait encoder and recognition head via trainable G2L and L2G adapter modules to improve gait recognition on four datasets.
Problem
Sequence-based gait recognition relies on expert-designed temporal architectures and dataset-specific features; the paper asks whether the sequence modeling capacity of LLMs can be transferred to gait sequences.
Approach
Treat gait sequences as "gait sentences": a G2L tokenizer (spatial transformer tokenizer plus peak-finding and cross-attention clip merging) converts frame features into clip-level tokens fed to a frozen LLM, and an L2G projector with global max pooling maps LLM outputs back to the gait feature space for fusion with the encoder features.
Key results
- GaitLLM-10 (DeepGaitV2-P3D encoder) improves Gait3D rank-1 from 67.4% to 74.1% over the 10-layer DeepGaitV2-P3D baseline, with mAP rising from 56.3% to 60.1%... (mAP 60.1 for P3D-10 variant)
- On GREW, GaitLLM with the 22-layer P3D encoder reaches 79.8% rank-1 versus 77.7% for the DeepGaitV2-P3D baseline.
- On CCPG, GaitLLM-10 reaches 85.4% mean rank-1 under the gait protocol versus 83.3% for DeepGaitV2-P3D.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add GaitLLM sequence modeling to DeepGaitV2-P3D 22-layer | DeepGaitV2-P3D 22-layer · Gait3D | rank-1 (%): 74.4 -> 76.5 | Helped |
| Add GaitLLM sequence modeling to DeepGaitV2-3D 22-layer | DeepGaitV2-3D 22-layer · GREW | rank-1 (%): 79.4 -> 80.3 | Helped |
| Add GaitLLM-10 | DeepGaitV2-P3D 10-layer · CCPG Gait Evaluation Protocol | Mean rank-1: 83.3 -> 85.4 | Helped |
| Add GaitLLM-10 | DeepGaitV2-P3D 10-layer · SUSTech1K | Overall rank-1: 82.3 -> 84.5 | Helped |
| Add the Spatial Tokenizer and Temporal Tokenizer (ablation) | GaitLLM-10 · Gait3D | rank-1 (%): 72.4 -> 74.1 | Helped |
| Add the Spatial Tokenizer (ablation) | GaitLLM-10 with Temporal Tokenizer · Gait3D | rank-1 (%): 73.5 -> 74.1 | Helped |
| Add the Temporal Tokenizer (ablation) | GaitLLM-10 with Spatial Tokenizer · Gait3D | rank-1 (%): 72.9 -> 74.1 | Helped |
| Replace the Temporal Tokenizer with average pooling (kernel 1, stride 1) (ablation) | GaitLLM-10 · Gait3D | rank-1 (%): 74.1 -> 72.9 | Hurt |
| Replace the Temporal Tokenizer with average pooling (kernel 4, stride 4) (ablation) | GaitLLM-10 · Gait3D | rank-1 (%): 74.1 -> 73.0 | Hurt |
| Replace the Temporal Tokenizer with average pooling (kernel 8, stride 8) (ablation) | GaitLLM-10 · Gait3D | rank-1 (%): 74.1 -> 72.7 | Hurt |
| Replace the Temporal Tokenizer with average pooling (kernel 16, stride 16) (ablation) | GaitLLM-10 · Gait3D | rank-1 (%): 74.1 -> 72.0 | Hurt |
| Use Global Max Pooling instead of Global Average Pooling (ablation) | GaitLLM-10 · Gait3D | rank-1 (%): 73.0 -> 74.1 | Helped |
Surprising findings
- Replacing Qwen 0.5B with GPT-2 still improves over no language model (72.5% vs 67.4% rank-1 on Gait3D), suggesting the gains are largely language-model-agnostic.
- Last-token aggregation performs worse than global average pooling and global max pooling on Gait3D (70.5% rank-1).
Limitations
- GaitLLM inference is much slower (31.5 FPS vs 75.3 FPS for the baseline) due to the LLM.
- Both gait encoder and LLM are frozen, so fine-tuning potential is unexplored.
- Gains depend on an off-the-shelf gait encoder; the paper notes no robust gait foundation model exists.
Experimental setup
Datasets: Gait3D GREW CCPG SUSTech1K
Benchmarks: Gait3D GREW CCPG SUSTech1K
Metrics: rank-1 accuracy rank-5 accuracy mAP mINP
Baselines: GaitSet GaitPart GaitGL GaitBase DANet QAGait VPNet-L DeepGaitV2-P3D DeepGaitV2-3D
Builds on: OpenGait DeepGaitV2 GaitBase LLaVA
Closest papers by meaning
- Vocabulary-Guided Gait Recognition (NeurIPS 2025 · similarity 0.90)
- BiggerGait: Unlocking Gait Recognition with Layer-wise Representations from Large Vision Models (NeurIPS 2025 · similarity 0.89)
- BigGait: Learning Gait Representation You Want by Large Vision Models (CVPR 2024 · similarity 0.87)
- Learning Visual Prompt for Gait Recognition (CVPR 2024 · similarity 0.86)
- Cross-Covariate Gait Recognition: A Benchmark (AAAI 2024 · similarity 0.85)
- RA-GAR: A Richly Annotated Benchmark for Gait Attribute Recognition (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.