Research map / Architectures and efficiency / Attention and transformers
Stochastic Transformer Networks With Linear Competing Units: Application To End-to-End SL Translation
Andreas Voskou, Konstantinos P. Panousis, Dimitrios Kosmopoulos, Dimitris N. Metaxas, Sotirios Chatzis
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace ReLU with LWTA activations (U=4) (ablation) | Bayesian Transformer for SLT · PHOENIX 2014T test | BLEU-4: 22.61 -> 23.65 | Improved |
| replace ELU with LWTA activations (U=4) (ablation) | Bayesian Transformer for SLT · PHOENIX 2014T test | BLEU-4: 22.56 -> 23.65 | Improved |
| replace SiLU with LWTA activations (U=4) (ablation) | Bayesian Transformer for SLT · PHOENIX 2014T test | BLEU-4: 22.33 -> 23.65 | Improved |
| use U=4 competing units instead of U=8 per block (ablation) | LWTA Transformer · PHOENIX 2014T test | BLEU-4: 22.96 -> 23.65 | Improved |
| use U=4 competing units instead of U=2 per block (ablation) | LWTA Transformer · PHOENIX 2014T test | BLEU-4: 22.82 -> 23.65 | Improved |
| use U=4 instead of U=16 per block (ablation) | LWTA Transformer · PHOENIX 2014T test | BLEU-4: 22.52 -> 23.65 | Improved |
| use variational Gaussian weights instead of point-estimates (ablation) | LWTA Transformer · PHOENIX 2014T test | BLEU-4: 22.34 -> 23.65 | Improved |
| use encoder-decoder depth 2-2 instead of 1-1 (ablation) | LWTA Transformer · PHOENIX 2014T test | BLEU-4: 21.85 -> 23.65 | Improved |
| use depth 2-2 instead of 3-3 (ablation) | LWTA Transformer · PHOENIX 2014T test | BLEU-4: 21.29 -> 23.65 | Improved |
| apply weight compression at inference (ablation) | LWTA Transformer 2-2 · PHOENIX 2014T test | BLEU-4: 23.65 -> 23.52 | No clear effect |
| apply weight compression at inference (ablation) | LWTA Transformer 3-3 · PHOENIX 2014T test | BLEU-4: 21.29 -> 20.77 | Hurt |
| use ensemble decoding with L=8 instead of single model | LWTA Transformer 2-2 · PHOENIX 2014T test | BLEU-4: 23.65 -> 25.59 | Improved |
Closest papers by meaning
- Scaling Sign Language Translation (NeurIPS 2024 · similarity 0.91)
- Conditional Variational Autoencoder for Sign Language Translation with Cross-Modal Alignment (AAAI 2024 · similarity 0.90)
- Sign2GPT: Leveraging Large Language Models for Gloss-Free Sign Language Translation (ICLR 2024 · similarity 0.89)
- Bridging Sign and Spoken Languages: Pseudo Gloss Generation for Sign Language Translation (NeurIPS 2025 · similarity 0.88)
- Improving Gloss-free Sign Language Translation by Reducing Representation Density (NeurIPS 2024 · similarity 0.88)
- ReSCUE: Re-translation with Sentence Commitment for Unsegmented Long-Form Simultaneous Sign Language Translation (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
