Research map / Image generation / Autoregressive and latent generation
GPS as a Control Signal for Image Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Fine-tunes a Stable Diffusion model on geotagged Flickr photos to generate images conditioned on GPS coordinates and text, and lifts 3D landmark NeRFs from angle-conditioned models via score distillation sampling.
Problem
Generating images that vary with precise GPS location within a city, which requires fine-grained knowledge of how appearance changes across space; prior work did not use GPS as a generative control signal.
Approach
Treat GPS tags from photo EXIF metadata as an additional conditioning signal in a diffusion model (embedded alongside CLIP text tokens and composed via classifier-free guidance), and use an azimuth-conditioned variant to guide score distillation sampling for NeRF reconstruction without explicit camera pose estimation.
Key results
- GPS-to-image model achieves FID 6.99 vs 11.26 for fine-tuned SD (text only) and 15.35 for SD (text), and GPS score 17.41 vs 9.43 for SD (text).
- Angle-to-image model reaches 22.36% angle accuracy vs 3.06% for Stable Diffusion and 2.78% random chance.
- 3D landmark reconstruction achieves CLIP score 31.87, perceptual quality 3.31, tourist score 3.45, outperforming DreamFusion (29.49, 2.21, 2.09).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Condition generation on GPS tags as well as text instead of fine-tuning on text alone | Stable Diffusion v1.4 GPS-to-image model · GPS-to-image test set | FID: 11.26 -> 6.99 | Helped |
| Condition generation on GPS tags as well as text instead of fine-tuning on text alone | Stable Diffusion v1.4 GPS-to-image model · GPS-to-image test set | GPS score: 11.53 -> 17.41 | Helped |
| Condition generation on GPS tags as well as text instead of fine-tuning on text alone | Stable Diffusion v1.4 GPS-to-image model · GPS-to-image test set | CLIP score: 32.12 -> 31.23 | Hurt |
| Condition generation on GPS tags as well as text instead of fine-tuning on text alone | Stable Diffusion v1.4 GPS-to-image model · GPS-to-image test set | OSV5M MAE: 28.49 -> 23.15 | Helped |
| Condition generation on GPS tags as well as text instead of fine-tuning on text alone | Stable Diffusion v1.4 GPS-to-image model · GPS-to-image test set | VQAScore: 82.97 -> 82.66 | Hurt |
| Condition angle-to-image generation on GPS-derived azimuth | Angle-to-image diffusion model · Angle-to-image evaluation | Angle accuracy: 3.06 -> 22.36 | Helped |
| Use GPS-conditioned angle-to-image diffusion rather than text-to-image DreamFusion for score distillation | 3D landmark reconstruction · 3D landmark reconstruction | CLIP score: 29.49 -> 31.87 | Helped |
| Use GPS-conditioned angle-to-image diffusion rather than text-to-image DreamFusion for score distillation | 3D landmark reconstruction · 3D landmark reconstruction | Perceptual Quality: 2.21 -> 3.31 | Helped |
| Use GPS-conditioned angle-to-image diffusion rather than text-to-image DreamFusion for score distillation | 3D landmark reconstruction · 3D landmark reconstruction | Tourist Score: 2.09 -> 3.45 | Helped |
Surprising findings
- The GPS-conditioned model's CLIP score (31.23) is slightly lower than the fine-tuned text-only baseline (32.12), a trade-off for location fidelity.
- COLMAP-based SfM reconstruction is all-or-nothing, succeeding on only 3 of 6 landmarks, while the GPS-guided method reconstructs scenes SfM cannot.
Limitations
- Approach may not work where few photos have GPS tags.
- Score distillation sampling produces saturated images.
- GPS tags can carry semantic information that is hard to disentangle from the image content.
Experimental setup
Datasets: Flickr geotagged photos (NYC Manhattan) Flickr geotagged photos (Paris) landmark photo sets (6)
Benchmarks: Manhattan and Paris held-out Flickr test set (30k photos per city) landmark angle accuracy 6-landmark 3D reconstruction
Metrics: FID CLIP score VQAScore GPS score OSV5M geolocation MAE/RMSE angle accuracy perceptual quality tourist score
Baselines: Stable Diffusion v1.4 (text+address) Stable Diffusion v1.4 (text) Fine-tuned SD (text) GPS nearest neighbor DreamFusion COLMAP + NeRF-W Nerfacto
Builds on: Stable Diffusion DreamFusion InstructPix2Pix classifier-free guidance BLIP-3
Closest papers by meaning
- GeoFidelity-Bench: Evaluating Block-Conditioned Geographic Fidelity of Street-View Generation (NeurIPS 2026 · similarity 0.86)
- GeoDiffusion: Text-Prompted Geometric Control for Object Detection Data Generation (ICLR 2024 · similarity 0.83)
- Controllable Satellite-to-Street-View Synthesis with Precise Pose Alignment and Zero-Shot Environmental Control (ICLR 2025 · similarity 0.83)
- G3: An Effective and Adaptive Framework for Worldwide Geolocalization Using Large Multi-Modality Models (NeurIPS 2024 · similarity 0.83)
- Weatherproofing Retrieval for Localization with Generative AI and Geometric Consistency (ICLR 2024 · similarity 0.83)
- Around the World in 80 Timesteps: A Generative Approach to Global Visual Geolocation (CVPR 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.