AI Paper Atlas

Research map / Video / Video-language understanding

MEGA: Multimodal Alignment Aggregation and Distillation For Cinematic Video Segmentation

Najmeh Sadoughi, Xinyu Li, Avijit Vajpayee, David Fan, Bing Shuai, Hector Santos-Villalobos, Vimal Bhat, Rohith MV

ICCV 2023 · Video › Video-language understanding · approach: Video LLMs with fine-grained understanding · 6 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add alignment positional encoding (ablation)MEGA · TRIPOD act segmentationTA (%): 5.29 -> 13.93Improved
add alignment positional encoding (ablation)MEGA · MovieNet-318 scene segmentationAP (%): 57.77 -> 58.59Improved
add alignment PE to bottleneck fusion tokens (V+T) (ablation)MEGA · TRIPODTA (%): 13.01 -> 14.63Improved
use aligned bottleneck fusion instead of late fusion (ablation)MEGA · TRIPOD act segmentationTA (%): 12.57 -> 13.93Improved
use multimodal synchronization with video features instead of screenplay-based synchronization (ablation)MEGA · TRIPOD act segmentationTA (%): 10.51 -> 13.93Improved
add BaSSL features to input (ablation)MEGA scene segmentation · MovieNet-318AP (%): 51.88 -> 58.59Improved
add CLIP_movie features to input (ablation)MEGA act segmentation · TRIPODTA (%): 6.09 -> 13.93Improved
add subtitle text features to visual input (ablation)MEGA act segmentation · TRIPODTA (%): 13.93 -> 14.63Improved
add audio features to MEGA (ablation)MEGA scene segmentation · MovieNet-318AP (%): 58.59 -> 55.36Hurt
add audio features (acoustic) to T+V act segmentation (ablation)MEGA · TRIPODTA (%): 14.63 -> 14.19Hurt
use 3 visual modalities instead of single modality (ablation)MEGA scene segmentation · MovieNet-318AP (%) / F1 (%): +0.67 AP, +4.59 F1Improved
expand GRAPHTP to same feature set Set2 as MEGA (ablation)GRAPHTP vs MEGA · TRIPODTA (%): 4.72 -> 14.19Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy