Research map / Large language models / Language, safety and interpretability
Two-Stream Network for Sign Language Recognition and Translation
Yutong Chen, Ronglai Zuo, Fangyun Wei, Yu Wu, Shujie LIU, Brian Mak
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add keypoint heatmap stream alongside RGB video (ablation) | TwoStream-SLR · Phoenix-2014T | WER: 22.42 -> 21.55 | Improved |
| Add bidirectional lateral connections (ablation) | TwoStream-SLR with video and keypoint encoders · Phoenix-2014T | WER: 21.55 -> 20.12 | Improved |
| Add sign pyramid network and auxiliary CTC losses (ablation) | TwoStream-SLR · Phoenix-2014T | WER: 20.12 -> 19.91 | Improved |
| Add a joint head (ablation) | TwoStream-SLR · Phoenix-2014T | WER: 19.91 -> 19.49 | Improved |
| Add frame-level self-distillation (ablation) | TwoStream-SLR · Phoenix-2014T | WER: 19.49 -> 19.32 | Improved |
| Use bidirectional instead of video-to-keypoint unidirectional lateral connections (ablation) | TwoStream-SLR · Phoenix-2014T dev | WER: 17.88 -> 17.72 | Improved |
| Apply lateral connections only at C2,C3 rather than C1,C2,C3 (ablation) | TwoStream-SLR · Phoenix-2014T test | WER: 19.32 -> 19.54 | Hurt |
| Apply SPN to both video and keypoint streams rather than one stream (ablation) | TwoStream-SLR · Phoenix-2014T test | WER: 19.39 -> 19.32 | Improved |
| Add auxiliary SPN supervision at P1 (ablation) | TwoStream-SLR · Phoenix-2014T test | WER: 19.32 -> 19.35 | Hurt |
| Use soft rather than hard distillation targets (ablation) | TwoStream-SLR · Phoenix-2014T test | WER: 19.58 -> 19.32 | Improved |
| Use the ensemble of three heads as teacher instead of the joint head (ablation) | TwoStream-SLR · Phoenix-2014T test | WER: 19.93 -> 19.32 | Improved |
| Distill into the joint head as well as video and keypoint heads (ablation) | TwoStream-SLR · Phoenix-2014T test | WER: 19.68 -> 19.32 | Improved |
Closest papers by meaning
- SignNet II: A Transformer-Based Two-Way Sign Language Translation Model (TPAMI 2022 · similarity 0.90)
- Lost in Translation, Found in Embeddings: Sign Language Translation and Alignment (NeurIPS 2026 · similarity 0.89)
- CVT-SLR: Contrastive Visual-Textual Transformation for Sign Language Recognition With Variational Alignment (CVPR 2023 · similarity 0.89)
- Scaling Sign Language Translation (NeurIPS 2024 · similarity 0.88)
- Uni-Sign: Toward Unified Sign Language Understanding at Scale (ICLR 2025 · similarity 0.88)
- Sign2GPT: Leveraging Large Language Models for Gloss-Free Sign Language Translation (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
