Research map / Image generation / Text-to-image diffusion

GeoFidelity-Bench: Evaluating Block-Conditioned Geographic Fidelity of Street-View Generation

NeurIPS 2026 · Image generation › Text-to-image diffusion · approach: Personalized and region-controlled generation

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces GeoFidelity-Bench, a benchmark of 112 named OSM street blocks in 25 cities with Mapillary reference panels, and evaluates six open-weight text-to-image models under city-only, named-block, and GPS-augmented prompts with same-city controls and retrieval-based fidelity metrics.

Problem

Text-to-image models produce plausible street scenes from place names, but existing metrics (FID, CLIPScore, LPIPS) do not show whether images resemble the requested block rather than a generic city stereotype.

Approach

Represent each named street block by a panel of real Mapillary images and evaluate generated images by similarity to the target panel and by retrieval against same-city and other-city alternative blocks, with prompt-specificity controls that swap in wrong street or neighborhood names.

Key results

• Street and neighborhood names raise mean CosSim by +0.036 (L1 minus L0, 95% CI [0.029, 0.042]) over 672 model-block pairs, while adding GPS coordinates adds only +0.004.
• Generated images retrieve their target block 39.9% of the time on average under named-block prompts, versus 85.5% for held-out real images.
• Exact-prompt CLIP alignment has weak association with geographic outcomes (retrieval Spearman 0.041) and near-chance AUROC (0.508) for all-four retrieval success.

Surprising findings

• Replacing only the street name leaves CosSim essentially unchanged (+0.001), while replacing the neighborhood or both names lowers it more.
• Random real images from the same country score higher CosSim (0.710) than every generator, yet have lower retrieval accuracy (0.259) than all generators.

Limitations

• Reference coverage depends on Mapillary contribution patterns and OSM naming density, with uneven city coverage (Tokyo 12%, New York 10% of images).
• GPS coordinates are supplied only as text, so L2 does not test coordinate-encoding models.
• Metrics depend on DINOv2 and Mask2Former encoders and measure visual agreement, not the current state of a street.

Experimental setup

Datasets: Mapillary OpenStreetMap Mapillary Vistas Cityscapes

Benchmarks: GeoFidelity-Bench

Metrics: CosSim DCSF MMD GAAS retrieval accuracy CLIP similarity AUROC Spearman correlation

Baselines: Held-out Real Random-Same-Country Random-Global CLIPScore

Builds on: CLIPScore DINOv2 Mapillary OpenStreetMap Maximum Mean Discrepancy

Closest papers by meaning

• GeoReasoner: Geo-localization with Reasoning in Street Views using a Large Vision-Language Model (ICML 2024 · similarity 0.87)
• GPS as a Control Signal for Image Generation (CVPR 2025 · similarity 0.86)
• UrBench: A Comprehensive Benchmark for Evaluating Large Multimodal Models in Multi-View Urban Scenarios (AAAI 2025 · similarity 0.86)
• Recognition through Reasoning: Reinforcing Image Geo-localization with Large Vision-Language Models (NeurIPS 2025 · similarity 0.85)
• Textual Supervision Enhances Geospatial Representations in Vision-Language Models (ICML 2026 · similarity 0.85)
• GRE Suite: Geo-localization Inference via Fine-Tuned Vision-Language Models and Enhanced Reasoning Chains (NeurIPS 2025 · similarity 0.85)