Research map / 3D vision / NeRF and radiance fields

DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Vision

CVPR 2024 · 3D vision › NeRF and radiance fields · approach: 3D-aware generative radiance fields · 140 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces DL3DV-10K, a 10,510-video 4K real-world multi-view scene dataset across 65 POI types, plus the DL3DV-140 NVS benchmark, benchmarking NeRF and 3DGS methods and showing pretraining generalizable NeRFs (IBRNet, MVSNeRF) on it helps.

Problem

Existing scene-level datasets for deep learning-based 3D vision are either synthetic or cover only a narrow set of real-world scenes, limiting comprehensive NVS benchmarking and learning of generalizable 3D priors.

Approach

Collect a large-scale, diverse set of real-world 4K multi-view videos from consumer mobile and drone cameras across 65 POI categories, annotate each scene with complexity indices (indoor/outdoor, light, reflection, transparency, texture frequency), and use a 140-scene subset as an NVS benchmark and the full set for pretraining generalizable NeRFs.

Key results

• On DL3DV-140, Zip-NeRF reaches 31.22 PSNR / 0.921 SSIM, 3DGS 29.82 PSNR / 0.919 SSIM, and Instant-NGP 25.01 PSNR.
• Pretraining MVSNeRF on DL3DV-2K raises DTU validation PSNR from 18.49 (from scratch) to 20.85.
• Pretraining IBRNet on DL3DV-2K raises Diffuse Synthetic 360 PSNR from 34.72 to 35.34, while ScanNet++ (270) pretraining slightly lowers it to 34.22.

Measured effects reported in the paper

Surprising findings

• Pretraining on ScanNet++ (270 scenes) did not improve IBRNet or MVSNeRF on their benchmarks, while DL3DV subsets did.
• Zip-NeRF's performance is sensitive to ray batch size: reducing it from 65536 to 4096 drops PSNR from 31.22 to 29.07.

Limitations

• Some scenes contain moving objects from mobile phone capture, adding challenges for NVS.
• Pilot generalizable NeRF experiments are limited to IBRNet and MVSNeRF on a small set of pretraining subsets.
• Outdoor unbounded scenes remain the hardest setting for all evaluated methods.

Experimental setup

Datasets: DL3DV-10K ScanNet++ DTU LLFF

Benchmarks: DL3DV-140 DTU LLFF Diffuse Synthetic 360

Metrics: PSNR SSIM LPIPS training time memory usage

Baselines: Nerfacto Instant-NGP Mip-NeRF 360 Zip-NeRF 3DGS IBRNet MVSNeRF

Builds on: NeRF Mip-NeRF 360 Zip-NeRF 3D Gaussian Splatting IBRNet

Closest papers by meaning

• From an Image to a Scene: Learning to Imagine the World from a Million 360° Videos (NeurIPS 2024 · similarity 0.88)
• The 3D-PC: a benchmark for visual perspective taking in humans and machines (ICLR 2025 · similarity 0.87)
• MegaSynth: Scaling Up 3D Scene Reconstruction with Synthesized Data (CVPR 2025 · similarity 0.86)
• A General Protocol to Probe Large Vision Models for 3D Physical Understanding (NeurIPS 2024 · similarity 0.86)
• MVSplat360: Feed-Forward 360 Scene Synthesis from Sparse Views (NeurIPS 2024 · similarity 0.86)
• OSN: Infinite Representations of Dynamic 3D Scenes from Monocular Videos (ICML 2024 · similarity 0.86)