Research map / Label-efficient and robust learning / Recognition and long-tail
SapiensID: Foundation for Human Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SapiensID, a single ViT-based human recognition model trained on the web-collected WebBody4M dataset that handles face and body recognition across varied poses and scales using Retina Patch, a masked recognition model, and a Semantic Attention Head.
Problem
Human recognition is usually handled by separate face and body models that rely on alignment or fixed camera setups, so they fail under varied pose, scale, and visibility.
Approach
Dynamic ROI-based patch tokenization (Retina Patch) keeps a fixed token budget per region of interest, masked training with attention scaling handles variable token counts, and a keypoint-queried attention head pools pose-robust part features, all trained with a margin-based softmax loss on a large web dataset.
Key results
- SapiensID reaches 73.05% average mAP on short-term ReID benchmarks and SoTA on all evaluated long-term ReID datasets with a single model.
- On the new WebBody-Test cross pose-scale setting, SapiensID gets 64.41% top-1 and 52.00% mAP-level average versus 47.12% top-1 for HAP trained on WebBody4M.
- Face recognition average of 97.31% across five aligned benchmarks, close to AdaFace's 97.63%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SapiensID trained on WebBody4M instead of HAP trained on WebBody4M | Short-Term ReID · Short-Term ReID (Table 1a average) | top1: 66.18 -> 72.01 | Helped |
| Use SapiensID instead of HAP | Long-Term ReID · Long-Term ReID (Table 1b average) | top1: 44.90 -> 66.30 | Helped |
| Use SapiensID instead of HAP trained on WebBody4M | WebBody-Test · WebBody-Test | top1: 47.12 -> 64.41 | Helped |
| Use SapiensID instead of KPR + SOLDIER | OccludedReID · OccludedReID | top1: 84.80 -> 87.30 | Helped |
| Use SapiensID instead of AdaFace-ViT | Face recognition · Face Avg | accuracy: 97.63 -> 97.31 | Hurt |
| Add Retina Patch and Semantic Attention Head to ViT (ablation) | SapiensID · Face Avg (CPLFW, CFPFP, CALFW, AGEDB) | accuracy: 90.63 -> 96.66 | Helped |
| Add Retina Patch and Semantic Attention Head to ViT (ablation) | SapiensID · Short-Term ReID average | accuracy: 56.17 -> 73.05 | Helped |
| Add Retina Patch and Semantic Attention Head to ViT (ablation) | SapiensID · Long-Term ReID average | accuracy: 31.81 -> 66.30 | Helped |
| Add Retina Patch to ViT (ablation) | SapiensID · Long-Term ReID average | accuracy: 31.81 -> 46.95 | Helped |
| Add Semantic Attention Head to ViT (ablation) | SapiensID · Short-Term ReID average | accuracy: 56.17 -> 72.63 | Helped |
| Replace learned mask token with MAE-style token dropping (ablation) | SapiensID · Long-Term ReID average | accuracy: 66.30 -> 64.46 | Hurt |
| Fix nk instead of sampling a variable masking rate (ablation) | SapiensID · Long-Term ReID average | accuracy: 66.30 -> 57.64 | Hurt |
Surprising findings
- Variable masking rate during training gives a larger benefit than the learned mask token choice in ablation.
- Lower body features add minimal or even negative value for PRCC, where upper body cues suffice.
Limitations
- Cross pose-scale ReID remains challenging with substantial room for improvement.
- Face recognition trails the specialized AdaFace model slightly.
- Occluded ReID mAP is lower than KPR despite higher top-1.
Experimental setup
Datasets: WebBody4M LTCC PRCC CCVID Market1501 MSMT17 CCDA Celeb-ReID OccludedReID LFW CPLFW CFP-FP
Benchmarks: LTCC PRCC CCVID Market1501 MSMT17 CCDA Celeb-ReID WebBody-Test OccludedReID LFW CPLFW CFP-FP
Metrics: top-1 mAP
Baselines: CAL CLIP3DReID SOLDIER HAP AdaFace KPR
Builds on: AdaFace Masked Autoencoders KP-RPE HAP
Closest papers by meaning
- SapiensID 2.0: Aligning Human Recognition Foundation Models with Human Perception (NeurIPS 2026 · similarity 0.92)
- ChatHuman: Chatting about 3D Humans with Tools (CVPR 2025 · similarity 0.86)
- CapHuman: Capture Your Moments in Parallel Universes (CVPR 2024 · similarity 0.86)
- MultiHuman-Testbench: Benchmarking Image Generation for Multiple Humans (NeurIPS 2025 · similarity 0.85)
- Referring Human Pose and Mask Estimation In the Wild (NeurIPS 2024 · similarity 0.84)
- Not Too Generative, Not Too Discriminative: The Human Alignment Sweet Spot (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.