Research map / 3D vision / 3D generation
Controllable Satellite-to-Street-View Synthesis with Precise Pose Alignment and Zero-Shot Environmental Control
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes IHA, an iterative homography adjustment of latent features during DDIM denoising, plus geometric cross-attention and CLIP-guided zero-shot text control, for satellite-to-street-view synthesis with better pose alignment and weather/illumination diversity.
Problem
Generating street-view images from satellite imagery with accurate pose alignment under a specified relative pose, and with controllable illumination and weather conditions that existing datasets lack.
Approach
Applies a homography correction to the latent at each DDIM step, with the homography updated by gradients from an InfoNCE pose-alignment loss between ground and satellite features; adds geometric cross-attention over height hypotheses and inference-time CLIP patch-text guidance for environmental control without retraining.
Key results
- On CVUSA, Ours achieves SSIM 0.38, FID 21.30 and depth 19.58, best on all reported metrics in Table 1.
- On VIGOR, Ours reaches SSIM 0.42, DINO 4.34 and depth 7.10, but ControlNet has better FID (23.68 vs 28.01).
- On KITTI ablation, adding GCA and IHA lowers average localization distance error from 10.74 to 10.51 m relative to the LDM baseline (modest gains).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add IHA to LDM (ablation) | LDM on KITTI · KITTI | Distance average: 10.74 -> 10.67 | Helped |
| Add IHA to LDM (ablation) | LDM on KITTI · KITTI | Angle average: 17.60 -> 17.46 | Helped |
| Add IHA to LDM (ablation) | LDM on KITTI · KITTI | Lateral d=1: 43.97 -> 44.08 | Helped |
| Add IHA to LDM (ablation) | LDM on KITTI · KITTI | Longitudinal d=1: 15.35 -> 15.77 | Helped |
| Add IHA to LDM (ablation) | LDM on KITTI · KITTI | Azimuth θ=1: 7.87 -> 8.08 | Helped |
| Add GCA to LDM with IHA (ablation) | LDM W. IHA on KITTI · KITTI | Distance average: 10.67 -> 10.51 | Helped |
| Add GCA to LDM with IHA (ablation) | LDM W. IHA on KITTI · KITTI | Angle average: 17.46 -> 17.14 | Helped |
| Add GCA to LDM with IHA (ablation) | LDM W. IHA on KITTI · KITTI | Lateral d=1: 44.08 -> 45.40 | Helped |
| Add GCA to LDM with IHA (ablation) | LDM W. IHA on KITTI · KITTI | Longitudinal d=1: 15.77 -> 16.78 | Helped |
| Add GCA to LDM with IHA (ablation) | LDM W. IHA on KITTI · KITTI | Azimuth θ=1: 8.08 -> 8.31 | Helped |
| Use IHA during inference (ablation) | LDM inference · IHA inference | Memory: 20126MB -> 21022MB | Hurt |
| Use IHA during inference (ablation) | LDM inference · IHA inference | Time Cost: 5.406s -> 5.513s | Hurt |
Surprising findings
- ControlNet achieves the best FID on VIGOR (23.68) yet is worse than the proposed method on the other geometric and semantic metrics.
- The authors report that pixel metrics such as RMSE and PSNR penalize clear, detailed generations and recommend against relying on them.
Limitations
- Building facade details are hard to reconstruct from satellite input.
- Lane markings in generated images are inconsistent with real ones, attributed to capture-time mismatch.
- Text-guided environmental control relies on CLIP patch similarity and is evaluated only with CLIP and BLIP scores.
Experimental setup
Datasets: CVUSA KITTI VIGOR
Benchmarks: CVUSA VIGOR KITTI
Metrics: SSIM self-similarity FID LPIPS (AlexNet and SqueezeNet) DINO similarity SegAny similarity RMSE PSNR SD depth error
Baselines: Pix2Pix XFork S2S Sat2Density ControlNet CrossDiff LDM
Builds on: Stable Diffusion DDIM ControlNet InfoNCE CCVPE
Closest papers by meaning
- Satellite to GroundScape - Large-scale Consistent Ground View Generation from Satellite Views (CVPR 2025 · similarity 0.86)
- MMM-RS: A Multi-modal, Multi-GSD, Multi-scene Remote Sensing Dataset and Benchmark for Text-to-Image Generation (NeurIPS 2024 · similarity 0.83)
- GPS as a Control Signal for Image Generation (CVPR 2025 · similarity 0.83)
- HORIZON: High-Resolution Semantically Controlled Panorama Synthesis (AAAI 2024 · similarity 0.83)
- StreetCrafter: Street View Synthesis with Controllable Video Diffusion Models (CVPR 2025 · similarity 0.83)
- DiffusionSat: A Generative Foundation Model for Satellite Imagery (ICLR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.