Research map / Vision-language and multimodal / Visual question answering
CoralVQA: A Large-Scale Visual Question Answering Dataset for Coral Reef Image Understanding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces CoralVQA, a 12,805-image, 277,653-QA coral reef VQA dataset built via a semi-automatic GPT-4o pipeline with expert verification, and benchmarks LVLMs on it.
Problem
Coral reef image interpretation requires domain expertise, and existing datasets lack genus-consistent annotations and multidimensional open-ended QA needed for training and evaluating vision-language models for coral monitoring.
Approach
A semi-automatic pipeline that cleans and re-annotates coral genus labels with marine biologists, extracts basic and ecological attributes, and uses GPT-4o with image input to generate QA pairs that are then human-verified across 16 dimensions.
Key results
- InternVL2.5 (fine-tuned) achieves the best test-set accuracy of 71.35% on basic visual attributes and 81.42% on ecological/health attributes, outperforming GPT-4o (54.94% and 43.42%).
- On the cross-region set, average accuracy drops to about 17-29% for fine-tuned models, e.g., InternVL2.5 reaches 18.75% (basic) and 28.25% (ecological).
- On the bleaching-coverage task, InternVL2.5 achieves MAE 0.0818 versus 0.1124 for Qwen2.5-VL.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune InternVL2.5 on CoralVQA training data | InternVL2.5 · CoralVQA test (basic visual) | accuracy: performance decreases of 43.68 % | Helped |
| Fine-tune InternVL2.5 on CoralVQA training data | InternVL2.5 · CoralVQA test (ecological and health) | accuracy: performance decreases of 35.54 % | Helped |
| Evaluate on an unseen marine region rather than the standard test region | Fine-tuned LVLMs · CoralVQA cross-region | accuracy: more than 30% | Hurt |
| Use InternVL2.5 instead of Qwen2.5VL | Bleaching-Coverage VQA · Bleaching-Coverage | MAE: 0.1124 -> 0.0818 | Helped |
| Use InternVL2.5 instead of Qwen2.5VL | Bleaching-Coverage VQA · Bleaching-Coverage | MASE: 1.2326 -> 0.8967 | Helped |
Surprising findings
- Closed-source GPT-4o and Claude3.5 Haiku perform poorly on coral-specific questions compared with fine-tuned open-source models.
- Fine-tuned models show a large accuracy drop on questions from unseen ocean regions, with nearly all models decreasing by over 30%.
Limitations
- Cross-region generalization is poor for all evaluated LVLMs.
- Bleaching-coverage benchmark is small (309 QA pairs).
- Some GPT-4o generated answers were inconsistent across repeated runs (average 8.2 of 10 consistent).
Experimental setup
Datasets: MLC XL Catlin Seaview Survey CoralsofttheWorld CoralVQA
Benchmarks: CoralVQA test set CoralVQA cross-region set Bleaching-Coverage
Metrics: accuracy MAE MASE
Baselines: GPT-4o Claude3.5 Haiku Mini-Gemini BLIP3 Qwen2.5-VL InternVL2.5
Builds on: Mini-Gemini BLIP3 Qwen2.5-VL InternVL2.5
Closest papers by meaning
- Enhancing Vision-Language Models with Morphological and Taxonomic Knowledge: Towards Coral Recognition for Ocean Health (AAAI 2025 · similarity 0.90)
- ReefNet: A Large-Scale Dataset and Benchmark for Fine-Grained Coral Reef Recognition (NeurIPS 2026 · similarity 0.90)
- CoralSCOP: Segment any COral Image on this Planet (CVPR 2024 · similarity 0.88)
- SeafloorAI: A Large-scale Vision-Language Dataset for Seafloor Geological Survey (NeurIPS 2024 · similarity 0.86)
- VLM4Bio: A Benchmark Dataset to Evaluate Pretrained Vision-Language Models for Trait Discovery from Biological Images (NeurIPS 2024 · similarity 0.86)
- Landsat30-AU: A Vision-Language Dataset for Australian Landsat Imagery (AAAI 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.