Research map / Vision-language and multimodal / Human and embodied modeling
SeafloorAI: A Large-scale Vision-Language Dataset for Seafloor Geological Survey
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces SeafloorAI, a large geo-distributed sonar image segmentation dataset from 62 USGS/NOAA surveys, and SeafloorGenAI, a GPT-4-generated vision-language extension, with UNet and TransUNet baselines showing large out-of-distribution degradation.
Problem
Machine learning for seafloor mapping from sonar imagery is limited by the scarcity of large, AI-ready, geographically diverse datasets, and no language-annotated sonar data exists for vision-language models.
Approach
Converts public hydrographic survey rasters and shapefiles into standardized, patchified multi-layer segmentation masks (Barnhardt and CMECS taxonomies) and extends them with GPT-4 in-context-generated descriptions and QA pairs validated by marine scientists.
Key results
- Baselines degrade sharply under geo-distributed shift: e.g., Sediment UNet-ResNet18 mIoU drops from 67.90 (ID) to 22.08 (OOD).
- Physio Zone UNet-Base OOD mIoU 43.22 vs ID 91.98; Habitat UNet-ResNet18 OOD mIoU 65.17.
- Human-expert review of GPT-4 annotations across prompt refinement iterations raised factual consistency and completeness accuracy over four iterations (chart values approximate).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use UNet-ResNet18 instead of UNet-Base | UNet · Sediment OOD | Acc: 21.49 ± 0.91 -> 34.71 ± 6.79 | Helped |
| Use UNet-ResNet18 instead of UNet-Base | UNet · Physio Zone OOD | Acc: 56.56 ± 0.87 -> 56.74 ± 2.53 | No clear effect |
| Use UNet-ResNet18 instead of UNet-Base | UNet · Habitat OOD | Acc: 70.54 ± 1.72 -> 76.40 ± 1.33 | Helped |
| Evaluate on target-region data instead of source-region data | UNet-Base · Sediment | Acc: 77.45 ± 0.81 -> 21.49 ± 0.91 | Hurt |
| Evaluate on target-region data instead of source-region data | UNet-Base · Physio Zone | Acc: 93.05 ± 0.16 -> 56.56 ± 0.87 | Hurt |
| Evaluate on target-region data instead of source-region data | UNet-Base · Habitat | Acc: 92.02 ± 0.18 -> 70.54 ± 1.72 | Hurt |
Surprising findings
- TransUNet, despite its larger architecture, has lower ID accuracy than the UNet variants on sediment and physio zone tasks.
- GPT-4 factual completeness of generated annotations was lower than factual consistency, attributed to hallucinations.
Limitations
- Layer availability is non-uniform across regions; e.g., Region 5 lacks Habitat, Fault and Fold layers.
- Habitat categories are coarse and exclude biotic classifications.
- The language component supports only straightforward analytical queries, not deeper reasoning; no VLM is trained in the paper.
Experimental setup
Datasets: USGS hydrographic surveys NOAA hydrographic surveys
Benchmarks: SeafloorAI multi-class segmentation (Sediment, Physio Zone, Habitat)
Metrics: pixel accuracy Dice coefficient Jaccard coefficient (mIoU)
Baselines: UNet-Base UNet-ResNet18 TransUNet-ViT-B/32
Builds on: LISA PixelLM TransUNet
Closest papers by meaning
- CoralVQA: A Large-Scale Visual Question Answering Dataset for Coral Reef Image Understanding (NeurIPS 2025 · similarity 0.86)
- ReefNet: A Large-Scale Dataset and Benchmark for Fine-Grained Coral Reef Recognition (NeurIPS 2026 · similarity 0.86)
- A Unified Image-Dense Annotation Generation Model for Underwater Scenes (CVPR 2025 · similarity 0.86)
- CoralSCOP: Segment any COral Image on this Planet (CVPR 2024 · similarity 0.85)
- PEACE: Empowering Geologic Map Holistic Understanding with MLLMs (CVPR 2025 · similarity 0.85)
- CoastalBench: A Decade-Long High-Resolution Dataset to Emulate Complex Coastal Processes (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.