Research map / Large language models / Language, safety and interpretability
Signs as Tokens: A Retrieval-Enhanced Multilingual Sign Language Generator
Ronglai Zuo, Rolandos Alexandros Potamias, Evangelos Ververas, Jiankang Deng, Stefanos Zafeiriou
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use multi-head decoding instead of sequential decoding (ablation) | SOKE without retrieval · How2Sign | DTW-PA-JPE avg: 4.64 -> 4.17 | Improved |
| use multi-head decoding instead of sequential decoding (ablation) | SOKE without retrieval · How2Sign | latency s/video: 3.26 -> 1.46 | Improved |
| use parallel decoding instead of sequential decoding (ablation) | SOKE without retrieval · How2Sign | DTW-PA-JPE avg: 4.64 -> 5.06 | Hurt |
| add sign dictionary retrieval (ablation) | SOKE multi-head · How2Sign | DTW-PA-JPE avg: 4.17 -> 3.34 | Improved |
| add sign dictionary retrieval (ablation) | SOKE multi-head · Phoenix-2014T | DTW-PA-JPE avg: 2.80 -> 2.13 | Improved |
| add sign dictionary retrieval (ablation) | SOKE multi-head · How2Sign | latency s/video: 1.46 -> 1.55 | No clear effect |
| train on multilingual data rather than monolingual (ablation) | SOKE · CSL-Daily | DTW avg: 4.18 -> 2.72 | Improved |
| train on multilingual data rather than monolingual (ablation) | SOKE · How2Sign | DTW avg: 4.14 -> 3.34 | Improved |
| add CSL-Daily data to How2Sign training (ablation) | SOKE · How2Sign | DTW body generation: 7.92 -> 7.11 | Improved |
| decouple body and hands into separate VQ-VAEs, then separate left and right hands (ablation) | SOKE tokenizer · How2Sign | DTW hand: 3.56 -> 3.13 -> 2.35 | Improved |
| set hand-token weight lambda to 1/3 instead of 0.1 (ablation) | SOKE multi-head decoding · How2Sign | DTW body: 7.95 -> 6.82 | Improved |
| increase hand codebook size from 128 to 192 (ablation) | SOKE tokenizer · How2Sign | JPE hand reconstruction: 7.07 -> 6.65 | Improved |
Closest papers by meaning
- LLMs are Good Sign Language Translators (CVPR 2024 · similarity 0.91)
- Scaling Sign Language Translation (NeurIPS 2024 · similarity 0.91)
- Lost in Translation, Found in Context: Sign Language Translation with Contextual Cues (CVPR 2025 · similarity 0.91)
- Advanced Sign Language Video Generation with Compressed and Quantized Multi-Condition Tokenization (NeurIPS 2025 · similarity 0.90)
- Bridging Sign and Spoken Languages: Pseudo Gloss Generation for Sign Language Translation (NeurIPS 2025 · similarity 0.90)
- KD-MSLRT: Lightweight Sign Language Recognition Model Based on Mediapipe and 3D to 1D Knowledge Distillation (AAAI 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
