Research map / 3D vision / Gaussian splatting

Coarse-to-Real: Generative Rendering for Populated Dynamic Scenes

NeurIPS 2026 · 3D vision › Gaussian splatting · approach: Monocular dynamic scene reconstruction · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes C2R, a two-stage generative renderer that turns coarse 3D crowd simulations into realistic urban videos by fine-tuning Wan 2.1 on real footage and then adding DINOv3-feature control trained with a small amount of paired synthetic data.

Problem

Traditional CG pipelines need expensive assets and lighting to render realistic populated urban crowd videos, while generative video models lack precise control over layout, camera motion, and crowd dynamics; paired coarse-to-real training data is also unavailable.

Approach

Decouple photorealistic prior learning (Stage I, fine-tuning on ~240k real clips) from control (Stage II, training a DINOv3-feature adapter added to diffusion latents) and use a heavily real-dominant mixture (99% real / 1% synthetic) with video-consistent HSV decorrelation to prevent appearance leakage from coarse inputs.

Key results

• Adopts a 99% real / 1% synthetic sampling ratio, which the authors select as best trade-off between VE-Bench controllability and VQAScore prompt adherence.
• Direct additive injection of DINO features (0 heads) achieves quality comparable to a single shared projection head, while 10 per-block heads overfit.

Surprising findings

• Direct feature addition with no extra projection performs comparably to a learned shared head, and per-block heads cause memorization of training inputs.
• Training on 100% real data gives the highest VE-Bench score but a sharp drop in VQAScore, indicating loss of generative flexibility.

Limitations

• Very sparse or abstract coarse 3D inputs may not provide enough structure to control layout or camera trajectory.
• Text prompts alone can be ambiguous for disambiguating camera motion or character movement.
• Non-autoregressive formulation prevents real-time or interactive generation.

Experimental setup

Datasets: 240k real street-view video clips 1.3k paired synthetic coarse-fine clips

Benchmarks: VE-Bench VQAScore

Metrics: VE-Bench structural controllability score VQAScore text-video alignment

Baselines: WAN VACE Control-A-Video Diffusion as Shader WAN+ControlNet Seedance 2.0

Builds on: Wan 2.1 ControlNet Flow Matching DINOv3 Adaptive Prompt Guidance

Closest papers by meaning

• DORSal: Diffusion for Object-centric Representations of Scenes $\textit{et al.}$ (ICLR 2024 · similarity 0.87)
• Generative Rendering: Controllable 4D-Guided Video Generation with 2D Diffusion Models (CVPR 2024 · similarity 0.87)
• Dynamic 3D Gaussian Fields for Urban Areas (NeurIPS 2024 · similarity 0.86)
• 4Real: Towards Photorealistic 4D Scene Generation via Video Diffusion Models (NeurIPS 2024 · similarity 0.86)
• TC-Light: Temporally Coherent Generative Rendering for Realistic World Transfer (NeurIPS 2025 · similarity 0.86)
• Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidance (NeurIPS 2025 · similarity 0.86)