Research map / Video / Video segmentation and tracking
SapiensID 2.0: Aligning Human Recognition Foundation Models with Human Perception
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Extends the SapiensID human recognition foundation model by distilling MLLM (Qwen2.5-VL-7B) soft-biometric semantics into its embedding space via subspace alignment and adding a keypoint-tracking temporal attention head (K-SAH) for video and gait recognition.
Problem
Human recognition foundation models rely on static geometric features, leading to semantic blindness (confusing people with similar clothing or differing fixed attributes) and missing temporal kinematic cues, while large annotated soft-biometric and video datasets are scarce.
Approach
Offline MLLM-generated fixed and dynamic soft-biometric text embeddings are used as teacher subspaces: ITA aligns principal components of fixed-trait embeddings, TND orthogonalizes features against dynamic-trait (clothing) subspaces, and K-SAH extends semantic cross-attention across a temporal window of keypoint-tracked patches, reusing image-pretrained weights.
Key results
- On CCVID general split, rank-1 improves from 92.57% (SapiensID) to 95.70%, and CC mAP from 72.22% to 78.04%.
- On CCPG clothing-change CL gait protocol, rank-1 rises from 33.9% to 60.8%.
- Long-term ReID average rises from 62.77% to 66.82%, while short-term average is slightly lower (71.31% vs 70.01% is higher; SOLIDER leads at 71.45%).
Surprising findings
- Gait models such as BiggerGait generalize strongly to video ReID on CCVID, achieving 85.13% rank-1 without ReID training data.
- The method trades a slight drop on short-term Market1501 and MSMT17 (88.18 to 86.94 top-1 on Market1501) for long-term robustness.
Limitations
- Semantic awareness is bounded by the zero-shot capability and biases of the teacher MLLM.
- Offline MLLM annotation of the full training corpus is a substantial one-time computational cost.
- Slight performance drop on some short-term ReID benchmarks where clothing is a valid cue.
Experimental setup
Datasets: WebBody4M WebBody CCVID CCPG LTCC PRCC Market1501 MSMT17 CCDA Celeb-ReID LFW CPLFW
Benchmarks: CCVID CCPG LTCC PRCC Market1501 MSMT17 CCDA Celeb-ReID WebBody LFW CPLFW CFP-FP
Metrics: rank-1 accuracy mAP face verification accuracy
Closest papers by meaning
- SapiensID: Foundation for Human Recognition (CVPR 2025 · similarity 0.92)
- MVHumanNet: A Large-scale Dataset of Multi-view Daily Dressing Human Captures (CVPR 2024 · similarity 0.84)
- Not Too Generative, Not Too Discriminative: The Human Alignment Sweet Spot (NeurIPS 2026 · similarity 0.84)
- Part-aware Unified Representation of Language and Skeleton for Zero-shot Action Recognition (CVPR 2024 · similarity 0.84)
- Neuron: Learning Context-Aware Evolving Representations for Zero-Shot Skeleton Action Recognition (CVPR 2025 · similarity 0.84)
- CapHuman: Capture Your Moments in Parallel Universes (CVPR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.