AI Paper Atlas

Research map / Video / Video segmentation and tracking

Read and Attend: Temporal Localisation in Sign Language Videos

Gul Varol, Liliane Momeni, Samuel Albanie, Triantafyllos Afouras, Andrew Zisserman

CVPR 2021 · Video › Video segmentation and tracking · approach: Sign language and physiological video analysis · 37 citations (OpenAlex)

Check your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Add attention-mined annotations to dictionary annotationsBSL-1K sign recognition · BSL-1K testrecnewtop-1 per-instance accuracy: 62.1 -> 64.2Improved
Add attention-mined annotations to mouthing annotationsBSL-1K sign recognition · BSL-1K testrecnewtop-1 per-instance accuracy: 40.8 -> 58.5Improved
Combine mouthing, dictionary, and attention annotationsBSL-1K sign recognition · BSL-1K testrecnewtop-1 per-instance accuracy: 63.5 -> 65.0Improved
Use greedy decoding with subtitle filtering instead of teacher forcingAttention-mined recognition annotations · BSL-1K testrecnewtop-1 per-instance accuracy: 38.7 -> 53.9Improved
Remove subtitle filtering from greedy-decoded predictions (ablation)Attention-mined recognition annotations · BSL-1K testrecnewtop-1 per-instance accuracy: 53.9 -> 50.6Hurt
Remove the requirement that training subtitles contain a sparse annotation (ablation)Video-to-text Transformer · BSL-1K testlocrecall: 16.5% vs 6.8%Hurt
Add sparse-annotation alignment loss (ablation)Video-to-text Transformer · BSL-1K testloclocalisation accuracy (teacher forcing): 51.1% -> 58.7%Improved
Add sparse-annotation alignment loss (ablation)Video-to-text Transformer · BSL-1K testlocrecognition top-1 per-instance accuracy: 53.9 vs 53.6No clear effect
Add curriculum from one-sign trimmed clips to subtitle-length inputs (ablation)Video-to-text Transformer · BSL-1K testlocrecall: 15.8 -> 16.0Improved
Use active-signer detection and sparse annotations to realign subtitles (ablation)Video-to-text Transformer · BSL-1K testlocrecall: 16.5% -> 15.4%Hurt
Include stop words in the subtitle vocabulary (ablation)Video-to-text Transformer · BSL-1K testlocrecall: 16.5 -> 13.9Hurt
Use original subtitle words rather than stemming (ablation)Video-to-text Transformer · BSL-1K testlocrecall: 16.5 -> 16.0No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy