AI Paper Atlas

Research map / Video / Video-language understanding

Learning to Ground Instructional Articles in Videos through Narrations

Effrosyni Mavroudi, Triantafyllos Afouras, Lorenzo Torresani

ICCV 2023 · Video › Video-language understanding · approach: Temporal video grounding · 23 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
train on wikiHow step pseudo-labels with direct pathway only (no narrations) (ablation)VINA · HT-Step valR@1: 30.7 -> 34.0 (+3.3 over TAN*)Improved
add iterative pseudo-label refinement to single-task step training (ablation)VINA · HT-Step valR@1 w/o nar.: 34.0 -> 35.8Improved
add multi-task training with narrations and steps (ablation)VINA · HTM-AlignR@1: 63.2 -> 64.8Improved
add multi-task training with narrations and steps, no narrations at inference (ablation)VINA · HT-Step valR@1 w/o nar.: 34.0 -> 34.3No clear effect
add iterative pseudo-labeling to multi-task model (ablation)VINA · HT-Step valR@1 w/ nar.: 36.1 -> 39.1Improved
fuse direct and indirect alignment at inference (ablation)VINA · HT-Step valR@1: 34.3 -> 36.1Improved
replace learned step-narration alignment with off-the-shelf MPNet similarity (ablation)indirect pathway · HT-Step valR@1: 30.5 -> 19.0Hurt
replace learned narration-to-video alignment with ASR timestamps (ablation)indirect pathway · HT-Step valR@1: 30.5 -> 27.9Hurt
use aggressive pseudo-label filtering (gamma 0.65, 91% discarded) instead of gamma 0.40 (ablation)iterative pseudo-labeling · HT-Step valR@1 w/ nar.: 36.5 -> 39.1Improved
use task ids predicted from narrations instead of HowTo100M metadata (ablation)article pairing · HT-Step valR@1: 34.3 -> 34.7No clear effect
add step positional embeddings and separate MLP (ablation)unimodal encoder · HT-Step val / HTM-AlignR@1: HT-Step w/o nar. 34.0 -> 34.3; HTM-Align 65.9 -> 64.8Mixed
use VINA instead of TANnarration alignment · HTM-AlignR@1: 49.4 -> 66.5Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy