AI Paper Atlas

Research map / Video / Video segmentation and tracking

Joint Visual and Audio Learning for Video Highlight Detection

Taivanbat Badamdorj, Mrigank Rochan, Yang Wang, Li Cheng

ICCV 2021 · Video › Video segmentation and tracking · approach: Self-supervised video representation learning · 91 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add disentangled attention instead of classic self-attention (ablation)Ours (AV) · VTWmAP: 0.808 -> 0.812Improved
add disentangled attention instead of classic self-attention (ablation)Ours (AV) · YouTube HighlightsmAP: 0.705 -> 0.718Improved
add noise sentinel to bimodal attention (ablation)Ours (AV) · VTWmAP: 0.804 -> 0.812Improved
add noise sentinel to bimodal attention (ablation)Ours (AV) · YouTube HighlightsmAP: 0.716 -> 0.718Improved
add audio via bimodal attention to visual-only self-attention modelSA (V) · VTWmAP: 0.722 -> 0.812Improved
add audio via bimodal attention to visual-only self-attention modelSA (V) · YouTube HighlightsmAP: 0.705 -> 0.718Improved
add audio via bimodal attention to visual-only self-attention modelSA (V) · TVSumtop-5 mAP: 0.7476 -> 0.7627Improved
use bimodal attention with noise sentinel instead of late fusion of self-attended featuresSA (AV)late · VTWmAP: 0.805 -> 0.812Improved
use bimodal attention with noise sentinel instead of early fusionSA (AV)early · YouTube HighlightsmAP: 0.703 -> 0.718Improved
use audio-only features instead of visual-only featuresSA · VTWmAP: 0.722 -> 0.794Improved
use audio-only features instead of visual-only featuresSA · TVSumtop-5 mAP: 0.748 -> 0.687Hurt
add audio-visual model over visual-only per categoryOurs (AV) · TVSum categoriestop-5 mAP: VU 0.6469 -> 0.5726; BT 0.9496 -> 0.9741Mixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy