Research map / Video / Video segmentation and tracking
EvSign: Sign Language Recognition and Translation with Streaming Events
Pengyu Zhang, Hao Yin, Zeren Wang, Wenyue Chen, Sheng Ming Li, Dong Wang, Huchuan Lu, Xu Jia
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use event streams instead of RGB frames for existing SLR methods on real EvSign data | VAC, CorrNet · EvSign test | WER (%): CorrNet 32.04 -> 29.95; VAC 30.43 -> 30.71 | Improved |
| use simulated events instead of RGB frames | VAC, TLP, SEN, CorrNet · PHOENIX14T test | WER (%): CorrNet 20.50 -> 24.55 | Hurt |
| add sparse backbone (SConv) to event baseline (ablation) | B(EV) · EvSign test | WER (%): 30.08 -> 30.01; FLOPS 238.88G -> 1.22G | No clear effect |
| add Local Token Fusion (ablation) | B(EV)+SConv · EvSign test | WER (%): 30.01 -> 29.48 | Improved |
| add Gloss-Aware Temporal Aggregation replacing BiLSTM (ablation) | B(EV)+SConv · EvSign test | WER (%): 30.01 -> 28.89 | Improved |
| add token aggregation before temporal modeling (ablation) | Ours S2G · EvSign test | WER (%): 40.34 -> 28.69 | Improved |
| average pooling instead of max pooling for token aggregation (ablation) | Ours S2G · EvSign test | WER (%): 33.24 -> 31.56 | Improved |
| LTF instead of 1D-CNN local aggregation (ablation) | Ours S2G · EvSign test | WER (%): 28.89 -> 28.69 | Improved |
| add gloss-aware mask attention (rho and delta) to global self-attention (ablation) | Ours S2G · EvSign test | WER (%): 31.14 -> 28.69 | Improved |
| soft mask instead of binarized hard mask in GAMA (ablation) | Ours S2G · EvSign test | WER (%): 29.31 -> 28.69 | Improved |
| use event input instead of RGB for SLT methods | SLT, VAC+TH, CorrNet+TH · EvSign test | ROUGE-L: SLT 40.05 -> 41.54; VAC+TH 39.08 -> 39.48 | Mixed |
| proposed model versus RGB SLT on synthetic PHOENIX14T | Ours S2GT · PHOENIX14T test | ROUGE-L: 40.26 (CorrNet+TH RGB) vs 40.21 | No clear effect |
Closest papers by meaning
- Lost in Translation, Found in Embeddings: Sign Language Translation and Alignment (NeurIPS 2026 · similarity 0.88)
- Scaling Sign Language Translation (NeurIPS 2024 · similarity 0.88)
- Bridging Sign and Spoken Languages: Pseudo Gloss Generation for Sign Language Translation (NeurIPS 2025 · similarity 0.88)
- Sign2GPT: Leveraging Large Language Models for Gloss-Free Sign Language Translation (ICLR 2024 · similarity 0.88)
- KD-MSLRT: Lightweight Sign Language Recognition Model Based on Mediapipe and 3D to 1D Knowledge Distillation (AAAI 2025 · similarity 0.87)
- Conditional Variational Autoencoder for Sign Language Translation with Cross-Modal Alignment (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
