Research map / Label-efficient and robust learning / Self-supervised and contrastive
SignBERT+: Hand-Model-Aware Self-Supervised Pre-Training for Sign Language Understanding
Hezhen Hu, Weichao Zhao, Wengang Zhou, Houqiang Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add self-supervised masked-pose pre-training (ablation) | SignBERT+ · MSASL (full) | per-instance Top-1: 53.31 -> 62.42 | Improved |
| add masked joint modeling (ablation) | SignBERT+ · MSASL (full) | per-instance Top-1: 53.31 -> 58.44 | Improved |
| add masked clip modeling (versus joint or frame only) (ablation) | SignBERT+ · MSASL (full) | per-instance Top-1: 53.31 -> 61.94 | Improved |
| combine joint, frame and clip masking (ablation) | SignBERT+ · MSASL (full) | per-instance Top-1: 61.94 -> 62.42 | Improved |
| add temporal and spatial position encoding (ablation) | SignBERT+ · MSASL (full) | per-instance Top-1: 51.80 -> 62.42 | Improved |
| add spatial position encoding only versus temporal only (ablation) | SignBERT+ · MSASL (full) | per-instance Top-1: 52.97 -> 60.95 | Improved |
| use hand-model-aware decoder instead of 1-layer fc (ablation) | SignBERT+ · MSASL (full) | per-instance Top-1: 59.88 -> 62.42 | Improved |
| replace position encoding with temporal GCN (k=3) (ablation) | SignBERT+ · MSASL (full) | per-instance Top-1: 62.42 -> 60.21 | No clear effect |
| raise masking ratio from 20% to 40% (ablation) | SignBERT+ · MSASL (full) | per-instance Top-1: 59.52 -> 62.42 | Improved |
| raise masking ratio from 40% to 60% (ablation) | SignBERT+ · MSASL (full) | per-instance Top-1: 62.42 -> 59.80 | Hurt |
| use 3 Transformer layers instead of 2 (ablation) | SignBERT+ · MSASL (full) | per-instance Top-1: 59.20 -> 62.42 | Improved |
| increase pose theta dimension from 25 to 35 (ablation) | SignBERT+ · MSASL (full) | per-instance Top-1: 62.42 -> 61.60 | Hurt |
Later papers that compared with this method
1 later accepted paper reports SignBERT as a baseline in their results tables; 1 of them report a better number on most of the results they share with it.
- BEST: BERT Pre-training for Sign Language Recognition with Coupling Tokenization (AAAI 2023 · better on 8 of 8 shared results)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- SignBERT: Pre-Training of Hand-Model-Aware Representation for Sign Language Recognition (ICCV 2021 · similarity 0.98)
- Hand-Model-Aware Sign Language Recognition (AAAI 2021 · similarity 0.94)
- Uni-Sign: Toward Unified Sign Language Understanding at Scale (ICLR 2025 · similarity 0.89)
- Lost in Translation, Found in Embeddings: Sign Language Translation and Alignment (NeurIPS 2026 · similarity 0.88)
- KD-MSLRT: Lightweight Sign Language Recognition Model Based on Mediapipe and 3D to 1D Knowledge Distillation (AAAI 2025 · similarity 0.88)
- Improving Gloss-free Sign Language Translation by Reducing Representation Density (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
