Research map / Large language models / Language, safety and interpretability
A Simple Multi-Modality Transfer Learning Baseline for Sign Language Translation
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Progressively pretrain the visual encoder on Kinetics-400, WLASL, then Sign2Gloss (ablation) | Sign2Gloss on PHOENIX-2014T · PHOENIX-2014T test | WER: 28.06 -> 22.45 | Helped |
| Add Kinetics-400 pretraining (ablation) | Sign2Gloss visual encoder · PHOENIX-2014T test | WER: 28.06 -> 23.50 | Helped |
| Add WLASL pretraining after Kinetics-400 (ablation) | Sign2Gloss visual encoder · PHOENIX-2014T test | WER: 23.50 -> 22.45 | Helped |
| Add within-domain Sign2Gloss pretraining (ablation) | Sign2Text visual encoder · PHOENIX-2014T test | BLEU-4: 21.44 -> 28.39 | Helped |
| Pretrain the translation model on CC25 (ablation) | Gloss2Text · PHOENIX test | BLEU-4: 22.57 -> 26.70 | Helped |
| Use progressive CC25 then Gloss2Text pretraining (ablation) | Sign2Text translation network · PHOENIX test | BLEU-4: 21.36 -> 28.39 | Helped |
| Add Gloss2Text domain pretraining after CC25 pretraining (ablation) | Sign2Text translation network · PHOENIX test | BLEU-4: 26.95 -> 28.39 | Helped |
| Use gloss representations rather than S3D features as V-L Mapper input (ablation) | Sign2Text · PHOENIX test | BLEU-4: 20.62 -> 28.39 | Helped |
| Use gloss representations rather than gloss logits as V-L Mapper input (ablation) | Sign2Text · PHOENIX test | BLEU-4: 27.78 -> 28.39 | Helped |
| Downsample input videos to 1/2 (ablation) | Sign2Text · PHOENIX test | BLEU-4: 28.39 -> 25.59 | Hurt |
| Downsample input videos to 1/3 (ablation) | Sign2Text · PHOENIX test | BLEU-4: 28.39 -> 21.89 | Hurt |
| Change the number of unfrozen S3D blocks during joint training (ablation) | Sign2Text visual encoder · PHOENIX test | BLEU-4: 27.79; 27.82; 28.03; 28.39 | No clear effect |
Closest papers by meaning
- Scaling Sign Language Translation (NeurIPS 2024 · similarity 0.94)
- Sign2GPT: Leveraging Large Language Models for Gloss-Free Sign Language Translation (ICLR 2024 · similarity 0.93)
- Lost in Translation, Found in Context: Sign Language Translation with Contextual Cues (CVPR 2025 · similarity 0.92)
- Bridging Sign and Spoken Languages: Pseudo Gloss Generation for Sign Language Translation (NeurIPS 2025 · similarity 0.92)
- Improving Gloss-free Sign Language Translation by Reducing Representation Density (NeurIPS 2024 · similarity 0.92)
- Lost in Translation, Found in Embeddings: Sign Language Translation and Alignment (NeurIPS 2026 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.