Research map / 3D vision / 3D generation
AerialMegaDepth: Learning Aerial-Ground Reconstruction and View Synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds AerialMegaDepth, a hybrid dataset pairing Google Earth pseudo-synthetic aerial renderings with registered MegaDepth ground photos, and fine-tunes DUSt3R, MASt3R and ZeroNVS to improve zero-shot ground-aerial pose estimation and novel view synthesis.
Problem
Learning-based multi-view geometry and novel-view synthesis fail under extreme viewpoint change between aerial and ground images, largely due to the lack of co-registered aerial-ground training data.
Approach
Render pseudo-synthetic aerial images from Google Earth textured meshes at varying altitudes, reconstruct them, and register real crowd-sourced ground photos into the same coordinate frame, yielding a hybrid co-registered dataset for fine-tuning geometry models.
Key results
- DUSt3R fine-tuned on hybrid data raises RRA@5 from 5.20% to 55.96% on ground-aerial pairs.
- MASt3R fine-tuned on hybrid data reaches 49.54% RRA@5 versus 3.36% for the released model.
- For two ground images plus one aerial image, DUSt3R-GA + Hybrid reaches 56.10% RRA@15 versus 14.63% for the released model.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune DUSt3R on pseudo-synthetic ground-aerial data (ablation) | DUSt3R · Ground-aerial pairs | RRA@5°: 5.20 -> 31.28 | Helped |
| Add real ground images to pseudo-synthetic training data (ablation) | DUSt3R + PSynth · Ground-aerial pairs | RRA@5°: 31.28 -> 55.96 | Helped |
| Fine-tune DUSt3R on MatrixCity | DUSt3R · Ground-aerial pairs | RRA@5°: 5.20 -> 17.85 | Helped |
| Fine-tune MASt3R on hybrid real and pseudo-synthetic data (ablation) | MASt3R · Ground-aerial pairs | RRA@5°: 3.36 -> 49.54 | Helped |
| Fine-tune ZeroNVS on AerialMegaDepth (ablation) | ZeroNVS · Pseudo-synthetic images | PSNR: 10.847 -> 12.381 | Helped |
| Fine-tune ZeroNVS on AerialMegaDepth (ablation) | ZeroNVS · Real images | PSNR: 7.478 -> 8.220 | Helped |
| Add one aerial reference image to two ground images and fine-tune on hybrid data (ablation) | DUSt3R-GA · Ground-camera pose, 2 ground images | RRA@15°: 12.20 -> 56.10 | Helped |
| Add one aerial reference image to two ground images and fine-tune on hybrid data (ablation) | MASt3R-SfM · Ground-camera pose, 2 ground images | RRA@15°: 9.03 -> 51.07 | Helped |
Surprising findings
- Pseudo-synthetic-only fine-tuning helps but adding real ground-level MegaDepth images roughly doubles the RRA@5 gain, indicating the mesh domain gap matters.
Limitations
- Pseudo-synthetic renderings from Google Earth lack ground-level visual fidelity and show facade artifacts and domain gap.
- Novel view synthesis remains very challenging; results are far from perfect at extreme viewpoints.
- Evaluation focuses on a limited set of six calibrated sites.
Experimental setup
Datasets: MegaDepth MegaScenes Google Earth pseudo-synthetic renderings MatrixCity ULTRRA Challenge ACCNVS1 WxBS
Benchmarks: ULTRRA Challenge ACCNVS1 WxBS
Metrics: RRA@5 RRA@10 RRA@15 RTA@5 RTA@10 RTA@15 pointmap accuracy delta@0.5m/1m/2m DreamSim LPIPS PSNR
Baselines: DUSt3R MASt3R LoFTR SuperPoint+SuperGlue MatrixCity fine-tuning ZeroNVS (MegaScenes)
Builds on: DUSt3R MASt3R ZeroNVS MegaDepth MegaScenes
Closest papers by meaning
- Horizon-GS: Unified 3D Gaussian Splatting for Large-Scale Aerial-to-Ground Scenes (CVPR 2025 · similarity 0.88)
- Seeing Across Skies and Streets: Feedforward 3D Reconstruction from Satellite, Drone, and Ground Images (NeurIPS 2026 · similarity 0.86)
- MV-DUSt3R+: Single-Stage Scene Reconstruction from Sparse Views In 2 Seconds (CVPR 2025 · similarity 0.86)
- DUSt3R: Geometric 3D Vision Made Easy (CVPR 2024 · similarity 0.85)
- MUSt3R: Multi-view Network for Stereo 3D Reconstruction (CVPR 2025 · similarity 0.85)
- Towards In-the-wild 3D Plane Reconstruction from a Single Image (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.