Research map / Architectures and efficiency / RNNs and normalization
Geo-Sign: Hyperbolic Contrastive Regularisation for Geometrically Aware Sign Language Translation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Geo-Sign projects ST-GCN skeletal features into a learnable-curvature Poincaré ball and uses a hyperbolic contrastive regulariser (pooled Fréchet-mean or token-attention variant) to regularise an mT5 translator, achieving gains over pose-only Uni-Sign on CSL-Daily.
Problem
Euclidean skeletal representations fed to LLM-based sign language translators blur fine-grained finger articulations relative to global body motion, limiting discriminability for SLT from pose data.
Approach
Map part-specific skeletal features into a hyperbolic Poincaré ball with learnable curvature and align them to text embeddings via a geodesic-distance InfoNCE-style contrastive loss, used as a regulariser on mT5 rather than as a RGB encoder replacement.
Key results
- On CSL-Daily test, Geo-Sign (Hyperbolic Token) reaches 27.42 BLEU-4 and 57.95 ROUGE-L, +1.81 BLEU-4 and +3.03 ROUGE-L over Uni-Sign pose-only (25.61 / 54.92).
- On How2Sign test, Geo-Sign (Token) gets 15.1 BLEU-4 vs 14.5 for Uni-Sign pose, but ROUGE-L 35.4 is below Uni-Sign Pose+RGB (36.0) and SSVP-SLT leads.
- On WLASL2000, Geo-Sign (Token) gets 63.64 per-instance and 61.89 per-class Top-1, small gains over Uni-Sign (Pose) 63.13 / 60.90.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add hyperbolic token-level contrastive regularisation | Uni-Sign pose model · CSL-Daily test | BLEU-4: 25.61 -> 27.42 | Helped |
| Use token-level instead of pooled global alignment (ablation) | Geo-Sign with hyperbolic regularisation · CSL-Daily test | BLEU-4: 27.17 -> 27.42 | Helped |
| Use hyperbolic rather than Euclidean token-level regularisation (ablation) | Geo-Sign · CSL-Daily test | BLEU-4: 25.98 -> 27.42 | Helped |
| Use token-level alignment rather than pooled alignment (ablation) | Geo-Sign with Euclidean regularisation · CSL-Daily test | BLEU-4: 25.72 -> 25.98 | Helped |
| Add hyperbolic token-level regularisation | Uni-Sign pose model · How2Sign test | BLEU-4: 14.5 -> 15.1 | Helped |
| Add Geo-Sign token method | Uni-Sign pose model · WLASL2000 test | Per-instance Top-1 accuracy: 63.13 -> 63.64 | Helped |
| Add Geo-Sign token method | Uni-Sign pose model · WLASL2000 test | Per-class Top-1 accuracy: 60.90 -> 61.89 | Helped |
| Set initial curvature c to 1.50 rather than 0.00 (ablation) | Geo-Sign (Hyperbolic Token) · CSL-Daily test | BLEU-4: 25.98 -> 27.42 | Helped |
| Set initial curvature c to 2.00 rather than 1.50 (ablation) | Geo-Sign (Hyperbolic Token) · CSL-Daily test | ROUGE-L: 57.95 -> 58.08 | Helped |
| Set initial curvature c to 2.00 rather than 1.50 (ablation) | Geo-Sign (Hyperbolic Token) · CSL-Daily test | BLEU-4: 27.42 -> 27.25 | Hurt |
| Set loss blending factor alpha to 0.70 rather than 0.10 (ablation) | Geo-Sign (Hyperbolic Token) · CSL-Daily test | BLEU-4: 25.74 -> 27.42 | Helped |
| Add Gaussian pose noise at level 0.01 (ablation) | Geo-Sign compared with Uni-Sign · CSL-Daily test | BLEU-4: 26.30 vs 24.14 | Helped |
Surprising findings
- Euclidean regularisation with the pooled method slightly underperforms Uni-Sign pose baseline on BLEU-4 (25.72 vs 25.61 is higher, but ROUGE-L on test is 55.57 vs 54.92), while the token variant's ROUGE-L drops to 53.93 on test despite BLEU gains.
- The Hyperbolic Token method's ROUGE-L on How2Sign trails the RGB+pose fusion baseline, and the paper notes the method is marginally worse than the best RGB method on How2Sign.
Limitations
- Gains over pose baseline are modest (e.g., +0.12 and +0.57 on WLASL2000) and results are single-run with minimal hyperparameter tuning on the dev set.
- Hyperbolic token alignment incurs additional memory and inference costs compared with the pooled variant.
- Gloss-free SOTA claim on ROUGE is limited to CSL-Daily; How2Sign ROUGE-L trails RGB fusion.
Experimental setup
Datasets: CSL-Daily How2Sign WLASL2000 CSL-News YTASL
Benchmarks: CSL-Daily How2Sign WLASL2000
Metrics: BLEU-1 BLEU-4 ROUGE-L Top-1 accuracy (per-instance, per-class)
Baselines: Uni-Sign (Pose) Uni-Sign (Pose+RGB) Euclidean regularisation (pooled/token) CV-SLT TS-SLT SignMusketeers SSVP-SLT NLA-SLR Sign-Rep
Builds on: Uni-Sign ST-GCN mT5 InfoNCE RAdam
Closest papers by meaning
- Lost in Translation, Found in Embeddings: Sign Language Translation and Alignment (NeurIPS 2026 · similarity 0.88)
- VSNet: Focusing on the Linguistic Characteristics of Sign Language (CVPR 2025 · similarity 0.87)
- Neural Sign Actors: A Diffusion Model for 3D Sign Language Production from Text (CVPR 2024 · similarity 0.86)
- Improving Gloss-free Sign Language Translation by Reducing Representation Density (NeurIPS 2024 · similarity 0.86)
- Sign2GPT: Leveraging Large Language Models for Gloss-Free Sign Language Translation (ICLR 2024 · similarity 0.86)
- Scaling Sign Language Translation (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.