Research map / 3D vision / 3D generation

DynaVol: Unsupervised Learning for Dynamic Scenes through Object-Centric Voxelization

ICLR 2024 · 3D vision › 3D generation · approach: Learning 3D reconstruction from images · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

DynaVol learns object-centric 4D voxel grids with canonical-space deformation and volume slot attention inside a compositional NeRF, enabling unsupervised 3D dynamic scene decomposition, novel view synthesis, and editing.

Problem

Unsupervised object-centric decomposition of dynamic visual scenes is typically done in 2D and ignores 3D consistency; the paper targets 3D-consistent unsupervised decomposition of dynamic scenes from monocular or sparse multi-view video.

Approach

Object-centric voxelization: a 4D voxel grid storing per-object occupancy probabilities at each location, deformed via a canonical-space dynamics module, combined with volume slot attention over voxel features and a compositional NeRF renderer, trained in three stages (3D warmup, 3D-to-4D expansion via connected components, 4D optimization).

Key results

• On real-world HyperNeRF scenes, DynaVol averages 23.50 PSNR / 0.844 SSIM versus 22.23 / 0.787 for HyperNeRF.
• On synthetic scene decomposition, DynaVol (V=5) reaches 96.95 FG-ARI on 3Fall versus 3.74 for SAVi and 28.77 for uORF.
• Ablation shows removing the 4D voxel optimization stage drops 3ObjFall PSNR from 32.11 to 29.69 and FG-ARI from 96.95 to 92.03.

Measured effects reported in the paper

Surprising findings

• Pretrained SAM outperforms uORF on several scenes in FG-ARI, and SAVi scores very low (3-8 FG-ARI) on the synthetic scenes despite being a 2D video method pretrained on MOVi-A.
• Training 4D voxel optimization from scratch yields high PSNR on some scenes but very low FG-ARI (e.g., 20.82 on 6ObjFall), showing the warmup stage is crucial for decomposition.

Limitations

• Decomposing moving objects in real-world scenes may require additional geometric priors; space-time continuity alone can fail for objects in contact (e.g., hand and banana).
• Falls short of state-of-the-art renderers such as 4K4D on ENeRF-Outdoor because it prioritizes object-centric representation over rendering quality.
• Each scene is overfit individually, so it is not a generalizing model.

Experimental setup

Datasets: Kubric synthetic scenes HyperNeRF D2NeRF ENeRF-Outdoor MOVi-A CLEVR-567

Benchmarks: Kubric synthetic dynamic scenes (3ObjFall, 3ObjRand, 3ObjMetal, 3Fall+3Still, 6ObjFall, 8ObjFall) HyperNeRF real scenes D2NeRF Duck ENeRF-Outdoor

Metrics: PSNR SSIM FG-ARI

Baselines: D-NeRF DeVRF DeVRF-Dyn NeuralDiff HyperNeRF D2NeRF SAVi uORF SAM OCLR

Builds on: slot attention SAVi uORF DVGO D-NeRF

Closest papers by meaning

• Object-X: Learning to Reconstruct Multi-Modal 3D Object Representations (NeurIPS 2025 · similarity 0.87)
• DORSal: Diffusion for Object-centric Representations of Scenes $\textit{et al.}$ (ICLR 2024 · similarity 0.86)
• Decompositional Neural Scene Reconstruction with Generative Diffusion Prior (CVPR 2025 · similarity 0.86)
• Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splatting (ICLR 2024 · similarity 0.86)
• 4D3R: Motion-Aware Neural Reconstruction and Rendering of Dynamic Scenes from Monocular Videos (NeurIPS 2025 · similarity 0.86)
• VoxelSplat: Dynamic Gaussian Splatting as an Effective Loss for Occupancy and Flow Prediction (CVPR 2025 · similarity 0.86)