Research map / Large language models / Language, safety and interpretability
A Simple Baseline for Spoken Language to Sign Language Translation with 3D Avatars
Ronglai Zuo, Fangyun Wei, Zenggui Chen, Brian Mak, Jiaolong Yang, Xin Tong
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add L_unseen (rest-pose prior for unseen joints) to SMPLSign-X (ablation) | SMPLSign-X · Phoenix-2014T dev | BLEU-4: 22.57 -> 24.16 | Improved |
| add L_upright (ablation) | SMPLSign-X · Phoenix-2014T dev | BLEU-4: 23.10 -> 24.16 | Improved |
| add L_smooth temporal consistency loss (ablation) | SMPLSign-X · Phoenix-2014T dev | BLEU-4: 23.64 -> 24.16 | Improved |
| add the sign connector instead of direct concatenation (ablation) | Spoken2Sign system · Phoenix-2014T dev | BLEU-4: 20.69 -> 24.16 | Improved |
| stitch signs in 3D space instead of 2D (ablation) | sign connector · Phoenix-2014T dev | BLEU-4: 21.83 -> 24.16 | Improved |
| add ISLR-based sign retrieval instead of random sign selection (ablation) | Spoken2Sign system · Phoenix-2014T dev | BLEU-4: 22.25 -> 24.16 | Improved |
| predict co-articulation duration instead of fixed duration (ablation) | sign connector · Phoenix-2014T dev | L1 distance: 1.83 -> 1.04 | Improved |
| add 3D keypoint augmentation for training ISLR (ablation) | keypoint-based ISLR · WLASL | per-instance top-1 accuracy: 46.20 -> 47.66 | Improved |
| add 3D keypoint augmentation (ablation) | keypoint-based ISLR · MSASL | per-instance top-1 accuracy: 51.81 -> 53.10 | Improved |
| add side view alongside frontal view for Sign2Spoken (ablation) | TwoStream keypoint model · CSL-Daily dev | BLEU-4: 21.66 -> 23.14 | Improved |
| side view only instead of frontal view (ablation) | Sign2Spoken · Phoenix-2014T dev | BLEU-4: 24.16 -> 22.60 | Hurt |
| SMPLSign-X 3D estimation vs direct HRNet 2D keypoints | Spoken2Sign back-translation · CSL-Daily dev | BLEU-4: 22.14 -> 21.66 | Hurt |
Closest papers by meaning
- Sign2GPT: Leveraging Large Language Models for Gloss-Free Sign Language Translation (ICLR 2024 · similarity 0.90)
- Lost in Translation, Found in Context: Sign Language Translation with Contextual Cues (CVPR 2025 · similarity 0.90)
- LLMs are Good Sign Language Translators (CVPR 2024 · similarity 0.89)
- Bridging Sign and Spoken Languages: Pseudo Gloss Generation for Sign Language Translation (NeurIPS 2025 · similarity 0.89)
- Lost in Translation, Found in Embeddings: Sign Language Translation and Alignment (NeurIPS 2026 · similarity 0.89)
- Scaling Sign Language Translation (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
