NeurIPS 2024 accepted papers on Video

143 of the 4,560 papers accepted at NeurIPS 2024 (3.1%) are on Video, compared with 2.4% at NeurIPS 2023. They fall into 4 topics.

See NeurIPS 2024 in the interactive map

Working on something in this area? Describe your idea in scime atlas to see the closest accepted papers by meaning and how crowded the spot has become.

Topics and papers

Video generation · 55 papers

VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Time (40 citations)
Diffusion4D: Fast Spatial-temporal Consistent 4D generation via Video Diffusion Models (24 citations)
VidProM: A Million-scale Real Prompt-Gallery Dataset for Text-to-Video Diffusion Models (20 citations)
StoryDiffusion: Consistent Self-Attention for Long-Range Image and Video Generation (17 citations)
Evaluation of Text-to-Video Generation Models: A Dynamics Perspective (14 citations)
MambaTalk: Efficient Holistic Gesture Synthesis with Selective State Space Models (12 citations)
FIFO-Diffusion: Generating Infinite Videos from Text without Training (11 citations)
Tell What You Hear From What You See - Video to Audio Generation Through Text (11 citations)
4Diffusion: Multi-view Video Diffusion Model for 4D Generation (9 citations)
OmniTokenizer: A Joint Image-Video Tokenizer for Visual Generation (9 citations)
COVE: Unleashing the Diffusion Feature Correspondence for Consistent Video Editing (8 citations)
Frieren: Efficient Video-to-Audio Generation Network with Rectified Flow Matching (8 citations)

and 43 more in the interactive map.

Video segmentation and tracking · 44 papers

Lips Are Lying: Spotting the Temporal Inconsistency between Audio and Visual in Lip-Syncing DeepFakes (18 citations)
Learning Truncated Causal History Model for Video Restoration (16 citations)
Advancing Video Anomaly Detection: A Concise Review and a New Dataset (15 citations)
EEG2Video: Towards Decoding Dynamic Visual Perception from EEG Signals (12 citations)
NVRC: Neural Video Representation Compression (11 citations)
VFIMamba: Video Frame Interpolation with State Space Models (11 citations)
Generalizable Implicit Motion Modeling for Video Frame Interpolation (10 citations)
FactorizePhys: Matrix Factorization for Multidimensional Attention in Remote Physiological Sensing (9 citations)
iVideoGPT: Interactive VideoGPTs are Scalable World Models (8 citations)
On Learning Multi-Modal Forgery Representation for Diffusion Generated Video Detection (8 citations)
Toward Dynamic Non-Line-of-Sight Imaging with Mamba Enforced Temporal Consistency (8 citations)
SpeechForensics: Audio-Visual Speech Representation Learning for Face Forgery Detection (6 citations)

and 32 more in the interactive map.

Video-language understanding · 33 papers

ShareGPT4Video: Improving Video Understanding and Generation with Better Captions (56 citations)
LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding (49 citations)
One Token to Seg Them All: Language Instructed Reasoning Segmentation in Videos (30 citations)
Streaming Long Video Understanding with Large Language Models (19 citations)
HourVideo: 1-Hour Video-Language Understanding (13 citations)
HAWK: Learning to Understand Open-World Video Anomalies (12 citations)
MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understanding (9 citations)
Diffusion-Inspired Truncated Sampler for Text-Video Retrieval (8 citations)
VERIFIED: A Video Corpus Moment Retrieval Benchmark for Fine-Grained Video Understanding (8 citations)
Learnability Matters: Active Learning for Video Captioning (6 citations)
ReXTime: A Benchmark Suite for Reasoning-Across-Time in Videos (6 citations)
MECD: Unlocking Multi-Event Causal Discovery in Video Reasoning (5 citations)

and 21 more in the interactive map.

NeurIPS 2024 accepted papers on Video

Topics and papers

Video generation · 55 papers

Video segmentation and tracking · 44 papers

Video-language understanding · 33 papers

Action recognition · 11 papers

More