Research map / Video / Video-language understanding
Video-Text Alignment Model for Sign Language Translation
Junyi Hu, Zhewen He, Haomian Huang, Yi Fang, Aoxiang Yang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add position-aligned contrastive loss (ablation) | VTaMo · Phoenix-2014T test | BLEU-4: 22.56 -> 28.86 | Improved |
| add local OT alignment loss (ablation) | VTaMo · Phoenix-2014T test | BLEU-4: 23.77 -> 28.86 | Improved |
| add global EMD loss (ablation) | VTaMo · Phoenix-2014T test | BLEU-4: 26.21 -> 28.86 | Improved |
| add orthogonality constraint on global transform (ablation) | VTaMo · Phoenix-2014T test | BLEU-4: 28.13 -> 28.86 | Improved |
| add memory queue to global alignment (ablation) | VTaMo · Phoenix-2014T test | BLEU-4: 27.35 -> 28.86 | Improved |
| add scheduled activation of global loss (ablation) | VTaMo · Phoenix-2014T test | BLEU-4: 26.42 -> 28.86 | Improved |
| use attention pooling instead of max-pooling in temporal encoder (ablation) | VTaMo · Phoenix-2014T test | BLEU-4: 26.57 -> 28.86 | Improved |
| use attention pooling instead of max-pooling in temporal encoder (ablation) | VTaMo · Phoenix-2014T test | ROUGE-L: 55.23 -> 60.24 | Improved |
| replace VTaMo alignment losses with generic CLIP4Clip/VideoCLIP/X-CLIP objectives (ablation) | VTaMo pipeline · Phoenix-2014T test | BLEU-4: 28.86 -> 9.47 | Hurt |
| add VTaMo alignment losses to CLIP-ViT-only pipeline (ablation) | VTaMo pipeline · Phoenix-2014T test | BLEU-4: 10.45 -> 28.86 | Improved |
| add learnable null token to OT alignment (ablation) | VTaMo local alignment · Phoenix-2014T test (dev) | alignment peak probability: 0.61 -> 0.76 | Improved |
| add epsilon annealing to Sinkhorn (ablation) | VTaMo local alignment · Phoenix-2014T test (dev) | alignment entropy: 0.55 -> 0.18 | Improved |
Closest papers by meaning
- Conditional Variational Autoencoder for Sign Language Translation with Cross-Modal Alignment (AAAI 2024 · similarity 0.93)
- Lost in Translation, Found in Embeddings: Sign Language Translation and Alignment (NeurIPS 2026 · similarity 0.92)
- LLMs are Good Sign Language Translators (CVPR 2024 · similarity 0.92)
- Scaling Sign Language Translation (NeurIPS 2024 · similarity 0.91)
- Bridging Sign and Spoken Languages: Pseudo Gloss Generation for Sign Language Translation (NeurIPS 2025 · similarity 0.90)
- Sign2GPT: Leveraging Large Language Models for Gloss-Free Sign Language Translation (ICLR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
