Research map / 3D vision / 3D generation
CityDreamer: Compositional Generative Model of Unbounded 3D Cities
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
CityDreamer composes separate instance-oriented (building) and stuff-oriented (background) BEV volumetric neural fields, driven by a MaskGIT-based unbounded layout generator, and introduces OSM and GoogleEarth CityGen datasets for unbounded 3D city generation.
Problem
Generating realistic, multi-view consistent, unbounded 3D cities is hard because buildings share one semantic class but vary widely in appearance, and existing 3D scene generators assign them a single label, causing structural distortions.
Approach
Split city generation into a building-instance generator and a background generator, each a BEV volumetric renderer with different scene parameterizations (generative hash grid for background stuff, periodic sin/cos positional encoding for building facades), composited into one image; the layout is produced by a MaskGIT-based unbounded layout generator.
Key results
- CityDreamer reaches FID 97.38 and KID 0.096 versus 123.83 / 0.109 for PersistentNature and 213.56 / 0.216 for SceneDreamer on GoogleEarth.
- CityDreamer achieves the lowest camera error (0.060) and depth error (0.147) among compared methods, with CE 0.186 for SceneDreamer.
- In a 20-volunteer user study, CityDreamer scored significantly higher than all baselines on perceptual quality, 3D realism, and view consistency.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CityDreamer instead of SceneDreamer | 3D city generation · GoogleEarth evaluation set | FID: 213.56 -> 97.38 | Helped |
| Use CityDreamer instead of PersistentNature | 3D city generation · GoogleEarth evaluation set | KID: 0.109 -> 0.096 | Helped |
| Use CityDreamer instead of SceneDreamer | 3D city generation · GoogleEarth generated frames | DE: 0.152 -> 0.147 | Helped |
| Use CityDreamer instead of SceneDreamer | 3D city generation · GoogleEarth generated frames | CE: 0.186 -> 0.060 | Helped |
| Use the Unbounded Layout Generator instead of InfinityGAN | city layout generation · Generated city layouts | FID: 183.14 -> 124.45 | Helped |
| Use the Unbounded Layout Generator instead of IPSM | city layout generation · Generated city layouts | KID: 0.502 -> 0.123 | Helped |
| Add the Building Instance Generator (ablation) | CityDreamer · GoogleEarth evaluation set | FID: 213.56 -> 97.38 | Helped |
| Add building instance labels to the Building Instance Generator (ablation) | CityDreamer · GoogleEarth evaluation set | FID: 117.75 -> 97.38 | Helped |
| Use Global encoder + Hash for background and Local encoder + SinCos for building parameterization (ablation) | CityDreamer · GoogleEarth evaluation set | FID: 219.30 -> 97.38 | Helped |
| Use Local encoder + SinCos for background parameterization (ablation) | CityDreamer · GoogleEarth evaluation set | FID: 97.38 -> 107.63 | Hurt |
| Use Global encoder + Hash for building parameterization (ablation) | CityDreamer · GoogleEarth evaluation set | FID: 97.38 -> 219.30 | Hurt |
| Use Global encoder + Hash for both background and building parameterization (ablation) | CityDreamer · GoogleEarth evaluation set | FID: 97.38 -> 213.56 | Hurt |
Surprising findings
- Using HashGrid for building generation hurt building structure: the paper reports that hash-grid parameterization makes buildings less structurally reasonable, so SinCos positional encoding is used for buildings instead.
Limitations
- Layout generation extrudes voxels to heights, so concave geometries such as caves and tunnels cannot be modeled.
- Buildings are generated individually at inference, giving higher computation cost.
- InfiniCity was not open-sourced, so it is excluded from quantitative comparison.
Experimental setup
Datasets: OSM GoogleEarth
Benchmarks: GoogleEarth
Metrics: FID KID depth error camera error user study score
Baselines: SGAM PersistentNature SceneDreamer InfiniCity InfinityGAN IPSM
Builds on: SceneDreamer MaskGIT NeRF GANCraft
Closest papers by meaning
- Generative Gaussian Splatting for Unbounded 3D City Generation (CVPR 2025 · similarity 0.88)
- CymbaDiff: Structured Spatial Diffusion for Sketch-based 3D Semantic Urban Scene Generation (NeurIPS 2025 · similarity 0.86)
- Dynamic 3D Gaussian Fields for Urban Areas (NeurIPS 2024 · similarity 0.86)
- Outdoor Scene Extrapolation with Hierarchical Generative Cellular Automata (CVPR 2024 · similarity 0.86)
- 3D-SceneDreamer: Text-Driven 3D-Consistent Scene Generation (CVPR 2024 · similarity 0.86)
- SemCity: Semantic Scene Generation with Triplane Diffusion (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.