Research map / Image generation / Text-to-image diffusion
Text2City: One-Stage Text-Driven Urban Layout Regeneration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Text2City, a one-stage text-conditioned DDPM model that jointly regenerates road and building layouts for a target urban region, with a new 147K-region OSM-based text-layout dataset.
Problem
Regenerating urban road and building layouts from intuitive text descriptions instead of hand-crafted rules, while accounting for road-building interactions and surrounding context.
Approach
A one-stage joint regenerator built on conditioned DDPMs that alternates road and building streams, with an interactive & enhanced guidance module exchanging prior knowledge between roads and buildings, plus CLIP-based attribute guidance and attribute-, geometry- and pixel-level constraints.
Key results
- Text2City reaches SSIM 0.89 and FID 77.71 versus 0.72 and 120.62 for the best baseline SDv2.
- Road layout similarity drops to 19.37 (from 56.19 for SDv2) and building layout similarity to 0.24 (from 0.60 for SDv2).
- In a 20-participant user study, Text2City scores 3.38 on rationality, 3.08 on harmony and 3.61 on match, highest among methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Text2City instead of SDv2 | Text-driven urban layout regeneration · Text-driven urban layout regeneration | SSIM: 0.72 -> 0.89 | Helped |
| use Text2City instead of SDv2 | Text-driven urban layout regeneration · Text-driven urban layout regeneration | FID: 120.62 -> 77.71 | Helped |
| use Text2City instead of SDv2 | Text-driven urban layout regeneration · Text-driven urban layout regeneration | WD: 0.10 -> 0.04 | Helped |
| use Text2City instead of SDv2 | Text-driven urban layout regeneration · Text-driven urban layout regeneration | RLS: 56.19 -> 19.37 | Helped |
| use Text2City instead of SDv2 | Text-driven urban layout regeneration · Text-driven urban layout regeneration | BLS: 0.60 -> 0.24 | Helped |
| fine-tune CLIP on the urban layout dataset (ablation) | Text2City · Text-driven urban layout regeneration | SSIM: 0.65 -> 0.89 | Helped |
| add the T2A network to decouple layout descriptions (ablation) | Text2City · Text-driven urban layout regeneration | SSIM: 0.68 -> 0.89 | Helped |
| add the IEG module for joint optimization (ablation) | Text2City · Text-driven urban layout regeneration | SSIM: 0.76 -> 0.89 | Helped |
| add road self-enhancement (ablation) | Text2City · Text-driven urban layout regeneration | SSIM: 0.78 -> 0.89 | Helped |
| add the geometry-level constraint Lg (ablation) | Text2City · Text-driven urban layout regeneration | SSIM: 0.81 -> 0.89 | Helped |
Surprising findings
- Original pre-trained CLIP generalizes poorly to urban layouts, which motivated fine-tuning on the new dataset.
Limitations
- Long inference time due to denoising and joint optimization.
- Uncommon layout descriptions may not be guided well by CLIP.
- Cannot specify low-level controls such as road length, style or building number.
Experimental setup
Datasets: OpenStreetMap Greater London layouts (147K regions) Paris and Shanghai OSM data (for generalization)
Benchmarks: Greater London urban layout dataset (147K regions)
Metrics: SSIM FID Wasserstein distance road layout similarity building layout similarity user study ranking score
Baselines: CityEngine IPSM UrbanBrush TDANet StyleCLIP BD SDv2
Builds on: DDPM Blended Diffusion CLIP Sentence-BERT StyleCLIP
Closest papers by meaning
- Seeing is Improving: Visual Feedback for Iterative Text Layout Refinement (CVPR 2026 · similarity 0.84)
- SSMG: Spatial-Semantic Map Guided Diffusion Model for Free-Form Layout-to-Image Generation (AAAI 2024 · similarity 0.83)
- Image Regeneration: Evaluating Text-to-Image Model via Generating Identical Image with Multimodal Large Language Models (AAAI 2025 · similarity 0.83)
- Build-A-Scene: Interactive 3D Layout Control for Diffusion-Based Image Generation (ICLR 2025 · similarity 0.82)
- Check Locate Rectify: A Training-Free Layout Calibration System for Text-to-Image Generation (CVPR 2024 · similarity 0.82)
- Rethinking The Training And Evaluation of Rich-Context Layout-to-Image Generation (NeurIPS 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.