AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

Sub-Word Level Lip Reading With Visual Attention

K R Prajwal, Triantafyllos Afouras, Andrew Zisserman

CVPR 2022 · Large language models › Language, safety and interpretability · approach: Sign and cross-lingual translation · 108 citations (OpenAlex)

Check your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Add WordPiece tokens instead of character-level tokens (ablation)TM-seq2seq lip-reading model · LRS2 testWER (%): 41.0 -> 37.2; -3.8Improved
Add Visual Transformer Pooling (ablation)WordPiece TM-seq2seq lip-reading model · LRS2 testWER (%): 37.2 -> 30.9; -6.3Improved
Add language-model beam rescoring (ablation)VTP lip-reading model · LRS2 testWER (%): 30.9 -> 29.8; -1.1Improved
Add horizontal-flip test-time augmentation (ablation)VTP model with beam LM rescoring · LRS2 testWER (%): 29.8 -> 28.9; -0.9Improved
Use VTP instead of Global Average Pooling (ablation)Visual Speech Detection model · AVA ActiveSpeaker testmAP: 80.2 -> 88.2Improved
Use VTP on finer (H/4, W/4) feature maps rather than coarser maps (ablation)Lip-reading visual backbone · LRS2 testWER (%): 35.7 -> 33.8 -> 30.9Improved
Train on all n-gram subsequences at once rather than gradually increasing sequence length (ablation)Lip-reading training protocol · LRS2 testWER (%): 30.92; 30.91No clear effect
Use BPE instead of WordPiece tokenization (ablation)Lip-reading decoder · LRS2 and LRS3recognition performanceHurt
Use Linear Transformer in VTP instead of the original Transformer (ablation)VTP module · Lip-reading benchmarksrecognition performanceNo clear effect
Add small-rotation and color-jitter test-time augmentation (ablation)Lip-reading inference · Lip-reading benchmarksrecognition performanceNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy