Research map / Vision-language and multimodal / CLIP and zero-shot

ReefNet: A Large-Scale Dataset and Benchmark for Fine-Grained Coral Reef Recognition

NeurIPS 2026 · Vision-language and multimodal › CLIP and zero-shot · approach: Open-vocabulary alignment from VLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces ReefNet, a ~925K-annotation, WoRMS-aligned coral genus point-annotation dataset from 76 CoralNet sources plus a new Al-Wajh (Red Sea) test set, and benchmarks zero-shot, few-shot, full fine-tuning, and cross-source recognition with VLMs, MLLMs, and vision-only models.

Problem

Coral reef monitoring lacks large-scale, taxonomically consistent, fine-grained coral annotations, and models degrade under cross-site domain shift.

Approach

Aggregate and re-verify CoralNet point annotations under a unified WoRMS taxonomy, filter to a high-confidence 39-genus subset (92% expert agreement), and benchmark four settings (zero-shot, cross-domain few-shot, within-source, cross-source to an unseen Red Sea site).

Key results

• Zero-shot: BioCLIP2 achieves 23.5% macro recall on ReefNet test and 43.2% on Al-Wajh, far above the best MLLM (Qwen3-VL, 5.1% and 15.2%).
• Full fine-tuning: ViT-B/16 reaches 79.84% macro recall on ReefNet test; Qwen3-VL (LoRA) reaches 59.80% on Al-Wajh, the best cross-source score.
• Cross-domain few-shot: BioCLIP2 reaches 58.79% (ReefNet) and 67.83% (Al-Wajh) at 20-shot, versus 27.22% and 31.27% for the best MLLM.

Surprising findings

• Zero-shot MLLMs perform very poorly, with LLaVA-NeXT collapsing to predicting a single genus for nearly all inputs.
• Fine-tuned vision-only ViT-B/16 outperforms fine-tuned VLMs and MLLMs in-domain, and cross-source performance drops for all models relative to within-source.

Limitations

• Annotations are patch-based point labels, limiting spatial/semantic segmentation use.
• Expert verification is constrained by taxonomist availability; only ~8,962 annotations were reviewed.
• Cross-source evaluation covers only one site (Al-Wajh) with 13 overlapping genera, and class counts in tail groups are small.

Experimental setup

Datasets: ReefNet CoralNet TreeOfLife-200M WoRMS

Benchmarks: ReefNet test (within-source) Al-Wajh test (cross-source)

Metrics: macro recall macro precision macro F1 accuracy

Baselines: CLIP OpenCLIP SigLIP BioCLIP BioCLIP2 Qwen3-VL InternVL-3.5 MiniCPM-V LLaVA-NeXT LLaVA-OneVision ViT-B/16 DeiT-B/16

Builds on: CLIP BioCLIP BioCLIP2 LoRA CoralNet

Closest papers by meaning

• Enhancing Vision-Language Models with Morphological and Taxonomic Knowledge: Towards Coral Recognition for Ocean Health (AAAI 2025 · similarity 0.93)
• CoralVQA: A Large-Scale Visual Question Answering Dataset for Coral Reef Image Understanding (NeurIPS 2025 · similarity 0.90)
• CoralSCOP: Segment any COral Image on this Planet (CVPR 2024 · similarity 0.90)
• SeafloorAI: A Large-scale Vision-Language Dataset for Seafloor Geological Survey (NeurIPS 2024 · similarity 0.86)
• DivShift: Exploring Domain-Specific Distribution Shifts in Large-Scale, Volunteer-Collected Biodiversity Datasets (AAAI 2025 · similarity 0.84)
• Fish-Vista: A Multi-Purpose Dataset for Understanding & Identification of Traits from Images (CVPR 2025 · similarity 0.83)