AI Paper Atlas

Research map / Vision-language and multimodal / CLIP and zero-shot

Zero-shot keyword spotting for visual speech recognition in-the-wild

Themos Stafylakis, Georgios Tzimiropoulos

ECCV 2018 · Vision-language and multimodal › CLIP and zero-shot · approach: Open-vocabulary alignment from VLMs · 39 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add joint G2P decoder with auxiliary phoneme loss (ablation)KWS network · LRS2 unseen keywordsEER: 8.46% -> 6.46%Improved
use phoneme targets instead of grapheme targets in the auxiliary loss (ablation)Joint KWS network · LRS2 unseen keywordsEER: 7.26% -> 6.46%Improved
use phoneme targets instead of grapheme targets in the auxiliary loss (ablation)Joint KWS network · LRS2 unseen keywordsFAR at MDR=1%: 40.26% -> 20.11%Improved
train the encoder jointly with the KWS loss rather than only through the G2P decoder (ablation)KWS network · LRS2 unseen keywordsEER: 7.22% -> 6.46%Improved
train the encoder jointly with the KWS loss rather than only through the G2P decoder (ablation)KWS network · LRS2 unseen keywordsFAR at MDR=1%: 30.99% -> 20.11%Improved
train the encoder jointly with the KWS loss rather than only through the G2P decoder (ablation)KWS network · LRS2 unseen keywordsTop-1 ranking rate: 34.14% -> 34.14%No clear effect
use frame-level concatenation with BiLSTM stack instead of a whole-utterance video embeddingKWS network · LRS2 unseen keywordsEER: 32.09% -> 6.46%Improved
add G2P decoder with auxiliary loss (ablation)KWS network · LRS2 unseen keywordsTop-1 ranking rate: 23.71% -> 34.14%Improved
add G2P decoder with auxiliary loss (ablation)KWS network · LRS2 unseen keywordsTop-8 ranking rate: 55.90% -> 67.70%Improved
use phoneme targets instead of grapheme targets in the auxiliary loss (ablation)Joint KWS network · LRS2 unseen keywordsTop-1 ranking rate: 31.16% -> 34.14%Improved
use the proposed visual KWS network instead of an ASR-based keyword searchKWS system · LRS2 unseen keywordsTop-8 ranking rate: 37.57% -> 67.70%Improved
increase the minimum keyword length from 6 phonemes to 7 and 8Joint-G2P · LRS2 unseen keywordserror ratesImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy