Research map / Vision-language and multimodal / Multimodal LLMs
PEACE: Empowering Geologic Map Holistic Understanding with MLLMs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces GeoMap-Bench, a 3,864-question benchmark for geologic map understanding, and GeoMap-Agent, a GPT-4o-based agent with hierarchical extraction, expert-knowledge injection, and prompt-enhanced QA that reaches 0.811 overall versus 0.369 for GPT-4o.
Problem
Current MLLMs perform poorly on geologic map understanding because of high-resolution inputs, many interrelated components, and required domain knowledge.
Approach
A divide-and-conquer agent that detects map components and legend units to extract metadata region by region, injects domain knowledge from simulated expert consultants with external tools (GEE, earthquake and fault databases), and uses enhanced prompts for question answering, evaluated on a new benchmark spanning extracting, grounding, referring, reasoning, and analyzing.
Key results
- GeoMap-Agent achieves an overall score of 0.811 on GeoMap-Bench (0.833 USGS, 0.789 CGS), versus 0.369 for GPT-4o.
- Ablation: removing HIE and DKI drops overall score on USGS from 0.833 to 0.376 when no modules are used.
- GeoMap-Agent with GPT-4o-mini as base reaches 0.740 USGS and 0.659 CGS overall.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GeoMap-Agent instead of GPT-4o | Geologic map question answering · GeoMap-Bench All Sets | overall: 0.369 -> 0.811 | Helped |
| Add HIE, DKI, and PEQA (ablation) | GeoMap-Agent · GeoMap-Bench USGS Set | overall: 0.376 -> 0.833 | Helped |
| Add HIE, DKI, and PEQA (ablation) | GeoMap-Agent · GeoMap-Bench CGS Set | overall: 0.361 -> 0.789 | Helped |
| Add PEQA to HIE and DKI (ablation) | GeoMap-Agent · GeoMap-Bench USGS Set | overall: 0.739 -> 0.833 | Helped |
| Add PEQA to HIE and DKI (ablation) | GeoMap-Agent · GeoMap-Bench CGS Set | overall: 0.714 -> 0.789 | Helped |
| Add DKI and PEQA to HIE (ablation) | GeoMap-Agent · GeoMap-Bench USGS Set | overall: 0.704 -> 0.833 | Helped |
| Add DKI and PEQA to HIE (ablation) | GeoMap-Agent · GeoMap-Bench CGS Set | overall: 0.653 -> 0.789 | Helped |
| Use GPT-4o instead of GPT-4o-mini as the base model (ablation) | GeoMap-Agent · GeoMap-Bench USGS Set | overall: 0.740 -> 0.833 | Helped |
| Use GPT-4o instead of GPT-4o-mini as the base model (ablation) | GeoMap-Agent · GeoMap-Bench CGS Set | overall: 0.659 -> 0.789 | Helped |
| Reduce map resolution to 1/2 or 1/4 (ablation) | GeoMap-Agent without HIE or DKI · GeoMap-Bench USGS Set | overall: 0.376 -> 0.302 | Hurt |
| Reduce map resolution to 1/2 or 1/4 (ablation) | GeoMap-Agent without HIE or DKI · GeoMap-Bench CGS Set | overall: 0.361 -> 0.362; 0.361 -> 0.358 | No clear effect |
| Add the last three PEQA prompt enhancements (ablation) | GeoMap-Agent without HIE or DKI · GeoMap-Bench USGS Set | overall: 0.376 -> 0.428 | Helped |
Surprising findings
- Reducing input resolution does not hurt GPT-4o-based performance, so HIE gains come from divide-and-conquer rather than resolution reduction.
Limitations
- Reasoning tasks such as fault detection and lithology composition remain challenging (reasoning score about 0.59).
- Struggles to recognize rocks with complex patterns or similar legend colors, especially in CGS maps.
- Essay-type evaluation relies on a GPT-4o pairwise judge, which may be biased.
Experimental setup
Datasets: USGS geologic maps CGS geologic maps (National 1:200,000 Digital Geologic Map)
Benchmarks: GeoMap-Bench
Metrics: accuracy IoU pairwise LLM judge score
Baselines: GPT-4o GPT-4o-mini QWen-chat GLM-4v-9b Idefics-9b-instruct Cogvlm2-llama3-chat-19B Monkey-chat
Builds on: chain-of-thought YOLOv10 K2
Closest papers by meaning
- GeoMMBench and GeoMMAgent: Toward Expert-Level Multimodal Intelligence in Geoscience and Remote Sensing (CVPR 2026 · similarity 0.88)
- LithoBench: Benchmarking Large Multimodal Models for Remote-Sensing Lithology Interpretation (NeurIPS 2026 · similarity 0.88)
- FRIEDA: Benchmarking Multi-Step Cartographic Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.86)
- SeafloorAI: A Large-scale Vision-Language Dataset for Seafloor Geological Survey (NeurIPS 2024 · similarity 0.85)
- MapEval: A Map-Based Evaluation of Geo-Spatial Reasoning in Foundation Models (ICML 2025 · similarity 0.85)
- Earth-Agent: Unlocking the Full Landscape of Earth Observation with Agents (ICLR 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.