Research map / 3D vision / 3D generation
X-Scene: Large-Scale Driving Scene Generation with High Fidelity and Flexible Controllability
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes X-Scene, a diffusion framework that generates large-scale 3D driving scenes via LLM/RAG-enriched text and layout control, joint occupancy-image-video generation, and consistency-aware outpainting, lifted to 3DGS.
Problem
Large-scale 3D driving scene generation with spatial coherence, appearance fidelity, and flexible multi-level controllability remains underexplored; prior methods need manually defined large layouts or lack appearance.
Approach
Combines LLM+RAG text enrichment and scene-graph-to-layout diffusion with a unified occupancy→image→video diffusion pipeline, and extends scenes by triplane occupancy outpainting plus reference-conditioned image extrapolation, then reconstructs 3DGS.
Key results
- Occupancy generation (17-class): FID3D 258.8 vs 529.6 for UniScene (51.2% reduction).
- Multi-view image generation at 224x400: FID 11.29, best among compared methods (vs Glad 12.57).
- Video generation: FVD 179.7 vs 209.9 for DreamForge and 217.9 for MagicDrive; UniAD evaluation improves over baseline on all metrics.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add VAE deformable attention (ablation) | X-Scene occupancy generation at triplane resolution (50,50,16) · Occupancy generation | IoU: 64.7 -> 66.6 | Helped |
| add VAE deformable attention (ablation) | X-Scene occupancy generation at triplane resolution (100,100,16) · Occupancy generation | IoU: 84.9 -> 85.6 | Helped |
| add layout condition (ablation) | X-Scene occupancy generation · Occupancy generation, triplane resolution (100,100,16) | FID: 1584 -> 258.8 | Helped |
| add box condition (ablation) | X-Scene occupancy generation · Occupancy generation, triplane resolution (100,100,16) | FID: 271.4 -> 258.8 | Helped |
| add semantic map (ablation) | X-Scene image generation · Multi-view image generation | FID: 12.23 -> 11.29 | Helped |
| add depth map (ablation) | X-Scene image generation · Multi-view image generation | FID: 12.94 -> 11.29 | Helped |
| add perspective map (ablation) | X-Scene image generation · Multi-view image generation | FID: 16.87 -> 11.29 | Helped |
| add 3D position embedding (ablation) | X-Scene image generation · Multi-view image generation | FID: 11.38 -> 11.29 | Helped |
| add text description (ablation) | X-Scene image generation · Multi-view image generation | FID: 12.60 -> 11.29 | Helped |
Limitations
- Temporal horizon limited; longer sequences left to future work.
- No multi-agent interaction modeling in generated scenarios.
Experimental setup
Datasets: nuScenes Occ3D-nuScenes
Benchmarks: Occ3D-nuScenes nuScenes
Metrics: IoU mIoU IS FID KID precision recall F-score FVD mAP
Baselines: UniScene DynamicCity OccSora OccWorld OccLLama MagicDrive DreamForge Panacea BEVGen BEVControl DriveDreamer Drive-WM
Builds on: triplane representation RePaint Stable Diffusion 3D Gaussian Splatting UniScene
Closest papers by meaning
- UniScene: Unified Occupancy-centric Driving Scene Generation (CVPR 2025 · similarity 0.91)
- Generating Multimodal Driving Scenes via Next-Scene Prediction (CVPR 2025 · similarity 0.89)
- SceneCrafter: Controllable Multi-View Driving Scene Editing (CVPR 2025 · similarity 0.89)
- Genesis: Multimodal Driving Scene Generation with Spatio-Temporal and Cross-Modal Consistency (NeurIPS 2025 · similarity 0.89)
- SceneDiffuser++: City-Scale Traffic Simulation via a Generative World Model (CVPR 2025 · similarity 0.88)
- DriveScape: High-Resolution Driving Video Generation by Multi-View Feature Fusion (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.