Research map / Video / Video segmentation and tracking
C2ST: Cross-Modal Contextualized Sequence Transduction for Continuous Sign Language Recognition
Huaiwen Zhang, Zihang Guo, Yang Yang, Xin Liu, De Hu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add cross-modal context learning (gloss language model fused with visual features) (ablation) | C2ST (TLP + Swin-T baseline) · Phoenix-2014 | WER test (%): 19.9 -> 19.0 | Improved |
| add contextualized sequence transduction loss (ablation) | C2ST (TLP + Swin-T baseline) · Phoenix-2014 | WER test (%): 19.9 -> 19.1 | Improved |
| use both CCL and CST together (ablation) | C2ST · Phoenix-2014 | WER test (%): 19.9 -> 17.7 | Improved |
| replace CTC loss with conditional gloss decoder loss (CGD) (ablation) | C2ST · Phoenix-2014 | WER test (%): 19.0 -> 18.3 | Improved |
| add sequence-level transduction calibration to CGD loss (ablation) | C2ST · Phoenix-2014 | WER test (%): 18.3 -> 17.7 | Improved |
| add video-level fusion on top of chunk-level fusion (ablation) | C2ST · Phoenix-2014 | WER test (%): 18.8 -> 17.7 | Improved |
| chunk-level fusion alone (ablation) | C2ST · Phoenix-2014 | WER test (%): 19.1 -> 18.8 | No clear effect |
| use BERT-Base instead of GloVe or word embedding as gloss language model (ablation) | C2ST · Phoenix-2014 | WER test (%): 18.2 (GloVe) -> 17.7 | Improved |
| use larger Swin-S backbone instead of Swin-T (ablation) | C2ST · Phoenix-2014 | WER test (%): 17.7 -> 18.2 | Hurt |
| use Swin-T instead of ResNet18 backbone (ablation) | C2ST · Phoenix-2014 | WER test (%): 18.3 -> 17.7 | Improved |
| adopt Swin and BERT in C2ST instead of ResNet and word embedding | C2ST · Phoenix-2014 efficiency | Params / throughput: 60.6M -> 187.8M params; 15.3 -> 3.0 infer throughput | Hurt |
Closest papers by meaning
- Cross-Sentence Gloss Consistency for Continuous Sign Language Recognition (AAAI 2024 · similarity 0.92)
- TCNet: Continuous Sign Language Recognition from Trajectories and Correlated Regions (AAAI 2024 · similarity 0.89)
- Lost in Translation, Found in Context: Sign Language Translation with Contextual Cues (CVPR 2025 · similarity 0.89)
- Bridging Sign and Spoken Languages: Pseudo Gloss Generation for Sign Language Translation (NeurIPS 2025 · similarity 0.89)
- Conditional Variational Autoencoder for Sign Language Translation with Cross-Modal Alignment (AAAI 2024 · similarity 0.88)
- SCOPE: Sign Language Contextual Processing with Embedding from LLMs (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
