Research map / Video / Video segmentation and tracking
ImViD: Immersive Volumetric Videos for Enhanced VR Engagement
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces ImViD, a 5K/60FPS multi-view audio-video dataset captured with a moving 39-46 camera rig, and benchmarks 3DGS-based dynamic light field methods plus a training-free 6-DoF sound field pipeline.
Problem
Existing volumetric and dynamic novel view synthesis datasets lack full 360° scene capture, large 6-DoF interaction space, synchronized multimodal audio, and high resolution/frame-rate long-duration content needed for immersive VR.
Approach
A moving multi-camera rig on a mobile cart with hemispherical camera layout and time-code synchronization captures synchronized 5K/60FPS multi-view video and audio in both fixed and moving strategies, and a training-free pipeline renders 6-DoF light and sound fields from it.
Key results
- STG++ achieves the best test-view PSNR among tested methods, e.g., 31.240 on Scene1 Opera vs 28.482 for STG and 23.227 for 4DGS.
- 4DGS performs far worse on textured outdoor Scene6 Puppy (PSNR 18.121, LPIPS 0.711) than STG++ (PSNR 20.533, LPIPS 0.202).
- In a 21-expert user study, 61.90% rated auditory spatial perception as excellent and 90.46% found the generated spatial audio immersive.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add global, time-invariant color mapping across frames to STG (ablation) | STG on ImViD · Opera girl, Laboratory, and Puppy test views | PSNR: 28.482 -> 31.240; 26.306 -> 27.581; 20.497 -> 20.533 | Helped |
| Add global, time-invariant color mapping across frames to STG (ablation) | STG on ImViD · Opera girl, Laboratory, and Puppy test views | SSIM: 0.786 -> 0.799; 0.910 -> 0.916; 0.594 -> 0.597 | Helped |
| Add global, time-invariant color mapping across frames to STG (ablation) | STG on ImViD · Opera girl, Laboratory, and Puppy test views | LPIPS: 0.287 -> 0.277; 0.114 -> 0.107; 0.211 -> 0.202 | Helped |
Surprising findings
- 4DGS trained from the frame-0 sparse point cloud with limited point count performs significantly worse on texture-rich outdoor scenes.
- The moving-rig dynamic data was not calibrated because existing calibration methods produced errors and floaters, so only time-synchronized data is released for it.
Limitations
- Calibrated results are not provided for the moving-rig dynamic data due to calibration failures.
- Sound field pipeline assumes a single dominant omnidirectional sound source and is evaluated only by a user study.
- Benchmarks cover only 3 of the 7 scenes and rely on per-60-frame segment training.
Experimental setup
Datasets: ImViD
Benchmarks: ImViD
Metrics: PSNR SSIM LPIPS FPS storage size user study ratings
Baselines: 4DGS 4DRotorGS STG
Builds on: STG 4DGS 4DRotorGS 3DGS COLMAP
Closest papers by meaning
- IM-Portrait: Learning 3D-aware Video Diffusion for Photorealistic Talking Heads from Monocular VideosC (CVPR 2025 · similarity 0.84)
- Imagine360: Immersive 360 Video Generation from Perspective Anchor (NeurIPS 2025 · similarity 0.84)
- AV-Cloud: Spatial Audio Rendering Through Audio-Visual Cloud Splatting (NeurIPS 2024 · similarity 0.84)
- ViewPoint: Panoramic Video Generation with Pretrained Diffusion Models (NeurIPS 2025 · similarity 0.83)
- EgoVid-5M: A Large-Scale Video-Action Dataset for Egocentric Videos Generation (NeurIPS 2025 · similarity 0.83)
- RePerformer: Immersive Human-centric Volumetric Videos from Playback to Photoreal Reperformance (CVPR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.