Research map / Video / Video segmentation and tracking
OLMD: Orientation-aware Long-term Motion Decoupling for Continuous Sign Language Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OLMD, a single-cue CSLR framework that aggregates long-term motion (LMA), decouples features into horizontal and vertical components for orientation-aware purification, and couples them within and across stages, reaching SOTA WER on PHOENIX14, PHOENIX14-T and CSL-Daily.
Problem
Continuous sign language recognition struggles with multi-orientational and long-term motions, which current RGB-based CSLR models capture poorly (they analyze only 1-2 adjacent frames and treat all orientations as one).
Approach
A Long-term Motion Aggregation module built on sliding-window feature differences suppresses static background, then motion features are pooled into horizontal and vertical components that are purified by separate 2D-CNN blocks, and coupled back within and across stages.
Key results
- OLMD reaches 17.1% Dev and 17.2% Test WER on PHOENIX14, improving over the prior single-cue SOTA TCNet by 1.0% (Dev) and 1.7% (Test).
- On CSL-Daily, OLMD achieves 25.8% Dev and 24.7% Test WER, versus SignGraph's 27.3% and 26.4% and the multi-cue TwoStream-SLR's 25.4% and 25.3%.
- Ablation shows decoupling at stages 2, 3 and 4 yields 2.1% Dev and 2.3% Test gains over baseline; a context length of 9 gives the lowest WER at 17.1/17.2.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use OLMD instead of TCNet | PHOENIX14 · PHOENIX14 Dev and Test | WER: 18.1 -> 17.1; 18.9 -> 17.2 | Helped |
| Use OLMD instead of TwoStream-SLR | PHOENIX14 · PHOENIX14 Dev and Test | WER: 18.4 -> 17.1; 18.8 -> 17.2 | Helped |
| Use OLMD instead of CorrNet | PHOENIX14-T · PHOENIX14-T Dev and Test | WER: 18.9 -> 17.1; 20.5 -> 18.4 | Helped |
| Use OLMD instead of SignGraph | CSL-Daily · CSL-Daily Dev and Test | WER: 27.3 -> 25.8; 26.4 -> 24.7 | Helped |
| Add decoupling after Stages 2, 3, and 4 (ablation) | OLMD · PHOENIX14 Dev and Test | WER: 19.2 -> 17.1; 19.5 -> 17.2 | Helped |
| Add LMA to HMP and VMP (ablation) | OLMD · PHOENIX14 Dev and Test | WER: 18.9 -> 17.1; 19.4 -> 17.2 | Helped |
| Add HMP and VMP to LMA (ablation) | OLMD · PHOENIX14 Dev and Test | WER: 18.3 -> 17.1; 19.3 -> 17.2 | Helped |
| Set LMA context to 9 instead of None (ablation) | OLMD · PHOENIX14 Dev and Test | WER: 18.9 -> 17.1; 19.4 -> 17.2 | Helped |
| Replace non-cascaded OMP with cascaded OMP (ablation) | OLMD · PHOENIX14 Dev and Test | WER: 17.3 -> 17.1; 18.1 -> 17.2 | Helped |
| Use AvgPool decoupling instead of no decoupling (ablation) | OLMD · PHOENIX14 Dev and Test | WER: 18.2 -> 17.1; 18.5 -> 17.2 | Helped |
| Add stage and cross-stage coupling (ablation) | OLMD · PHOENIX14 Dev and Test | WER: 18.3 -> 17.1; 18.5 -> 17.2 | Helped |
| Add stage coupling alone (ablation) | OLMD without cross-stage coupling · PHOENIX14 Dev and Test | WER: 18.3 -> 17.9; 18.5 -> 18.3 | Helped |
Surprising findings
- On CSL-Daily Test, OLMD (24.7%) beats the multi-cue TwoStream-SLR (25.3%) despite using only RGB video.
Limitations
- Gains on PHOENIX14 Dev/Test are relatively modest versus some baselines in ablations; per-component improvements are small in places.
- Relies on long-term context windows that increase FLOPs and inference time as context grows.
Experimental setup
Datasets: PHOENIX14 PHOENIX14-T CSL-Daily
Benchmarks: PHOENIX14 PHOENIX14-T CSL-Daily
Metrics: WER deletion/insertion rate
Baselines: SFL VAC SMKD TLP SEN AdaBrowse+ CorrNet CoSign SignGraph TCNet TwoStream-SLR C2SLR
Builds on: VAC SMKD TLP LGD CTC
Closest papers by meaning
- Continuous Sign Language Recognition With Correlation Network (CVPR 2023 · similarity 0.86)
- TCNet: Continuous Sign Language Recognition from Trajectories and Correlated Regions (AAAI 2024 · similarity 0.86)
- Distilling Cross-Temporal Contexts for Continuous Sign Language Recognition (CVPR 2023 · similarity 0.86)
- VSNet: Focusing on the Linguistic Characteristics of Sign Language (CVPR 2025 · similarity 0.85)
- KD-MSLRT: Lightweight Sign Language Recognition Model Based on Mediapipe and 3D to 1D Knowledge Distillation (AAAI 2025 · similarity 0.84)
- Cross-Sentence Gloss Consistency for Continuous Sign Language Recognition (AAAI 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.