AI Paper Atlas

Research map / Video / Video-language understanding

Learning Modality Interaction for Temporal Sentence Localization and Event Captioning in Videos

Shaoxiang Chen, Wenhao Jiang, Wei Liu, Yu-Gang Jiang

ECCV 2020 · Video › Video-language understanding · approach: Dense video captioning · 100 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add interaction on concatenated multimodal features (ablation)captioning baseline · MSVDCIDEr: 62.57 -> 66.10Improved
use pairwise interaction instead of interacting on concatenated features (ablation)PMI-CAP · MSVDCIDEr: 66.10 -> 75.30Improved
use sum fusion instead of concatenation after pairwise interaction (ablation)PMI-CAP · MSVDCIDEr: 75.30 -> 78.42Improved
use learned importance-weighted fusion instead of sum fusion (ablation)PMI-CAP · MSVDCIDEr: 78.42 -> 95.17Improved
add inter-modality interactions to intra-modality only (ablation)PMI-CAP · MSVDCIDEr: 88.46 -> 95.17Improved
add intra-modality interactions to inter-modality only (ablation)PMI-CAP · MSVDCIDEr: 70.20 -> 95.17Improved
add intra-modality interaction alone to concat baseline (ablation)captioning baseline · MSVDCIDEr: 62.57 -> 88.46Improved
inter-modality interaction alone vs baseline (ablation)captioning baseline · MSVDCIDEr: 62.57 -> 70.20Improved
add PMI to the localizer (ablation)PMI-LOC · Charades-STAR@1 IoU=0.5: 35.34 -> 37.05Improved
add video-text local interaction (ablation)PMI-LOC · Charades-STAR@1 IoU=0.5: 37.05 -> 38.42Improved
add l2-norm regularization (ablation)PMI-LOC · Charades-STAR@1 IoU=0.5: 38.42 -> 39.73Improved
add l2-norm regularization (ablation)PMI-LOC · Charades-STAR@1 IoU=0.7: 18.63 -> 19.27Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy