Research map / Image generation / Autoregressive and latent generation
BioTrove: A Large Curated Image Dataset Enabling AI for Biodiversity
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces BioTrove, a 161.9M-image research-grade biodiversity dataset curated from iNaturalist, and BioTrove-CLIP models trained on a 40M-image subset with new zero-shot benchmarks.
Problem
Standard vision datasets under-cover biodiversity and agriculture species, and existing biodiversity image datasets are smaller and less AI-ready, limiting fine-grained and zero-shot recognition for ecological applications.
Approach
Curate a very large, research-grade, AI-ready image-text dataset from the iNaturalist Open Dataset with dual common/scientific-name and taxonomic-hierarchy captions, release a filtering/balancing pipeline, and train CLIP-style models (BioTrove-CLIP) on a 40M subset with semi-global shuffling.
Key results
- BT-CLIP-O reaches 91.1 ± 0.17% top-1 on BioTrove-Balanced, versus 7.3% for OpenAI CLIP ViT-B/16 on the same benchmark.
- BioCLIP-B scores 68.2% on BioTrove-Unseen and outperforms BT-CLIP-O (47.1%) there, which the authors attribute to species overlap with TreeOfLife-10M.
- BT-CLIP-M reaches the highest weighted average of 69.5 across benchmarks, versus 58.5 for BioCLIP-B and 14.7 for OpenAI-B; BT-CLIP-O scores 70.8 on this average, the best of all models.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune OpenAI ViT-B/16 on BioTrove-Train | OpenAI ViT-B/16 · Weighted Avg. | weighted average accuracy: 14.7 -> 70.8 | Helped |
| Fine-tune BioCLIP ViT-B/16 on BioTrove-Train | BioCLIP ViT-B/16 · Weighted Avg. | weighted average accuracy: 58.5 -> 67.2 | Helped |
| Fine-tune MetaCLIP ViT-L/14 on BioTrove-Train | MetaCLIP ViT-L/14 · Weighted Avg. | weighted average accuracy: 25.0 -> 69.5 | Helped |
| Fine-tune BioCLIP ViT-B/16 on BioTrove-Train | BioCLIP ViT-B/16 · BioTrove-Unseen | top-1 zero-shot accuracy: 68.2 -> 53.8 | Hurt |
| Fine-tune BioCLIP ViT-B/16 on BioTrove-Train | BioCLIP ViT-B/16 · BioCLIP-Rare | top-1 zero-shot accuracy: 30.2 -> 23.7 | Hurt |
| Fine-tune MetaCLIP ViT-L/14 on BioTrove-Train | MetaCLIP ViT-L/14 · Life-Stages | top-1 zero-shot accuracy: 64.3 -> 26.3 | Hurt |
| Train on BioTrove-Train rather than iNat21 alone | BioCLIP checkpoint · BioTrove-Unseen | Top-1 accuracy: 0.2100 -> 0.5380 | Helped |
| Train on BioTrove-Train rather than iNat21 alone | BioCLIP checkpoint · Fungi | Top-1 accuracy: 0.4420 -> 0.5390 | Helped |
| Train on BioTrove-Train rather than iNat21 alone | BioCLIP checkpoint · Life-Stages | Top-1 accuracy: 0.2867 -> 0.1500 | Hurt |
| Train on BioTrove-Train rather than iNat21 alone | BioCLIP checkpoint · DeepWeeds | Top-1 accuracy: 0.2057 -> 0.1840 | Hurt |
| Train on BioTrove-Train rather than iNat21 alone | BioCLIP checkpoint · Insects-2 | Top-1 accuracy: 0.0103 -> 0.1690 | Helped |
| Train on BioTrove-Train rather than iNat21 alone | BioCLIP checkpoint · Birds-525 | Top-1 accuracy: 0.5030 -> 0.5710 | Helped |
Surprising findings
- BioCLIP outperforms BioTrove-CLIP on BioTrove-Unseen and BioCLIP-Rare, which the authors attribute partly to possible forgetting during fine-tuning and species overlap with TreeOfLife.
- All models perform at or below random chance on the Confounding-Species benchmark, so it was not reported.
Limitations
- BioTrove-CLIP performance declines at taxonomic levels below species, and it underperforms OpenAI and MetaCLIP baselines on the life-stages benchmark.
- Fine-tuned models show mixed gains; BioCLIP remains superior on several datasets, likely because TreeOfLife-10M has more species classes than BioTrove-Train.
- Dataset contains offensive content (e.g., dead or dismembered animals) retained for scientific value and may include some harmful images.
Experimental setup
Datasets: BioTrove BioTrove-Train iNaturalist Open Dataset iNat2021 TreeOfLife-10M Birds-525 BioCLIP-Rare Fungi (Danish) DeepWeeds Insects-2 Confounding Species
Benchmarks: BioTrove-Unseen BioTrove-Balanced BioTrove-LifeStages BioCLIP-Rare Fungi Insects-2 Birds-525 DeepWeeds
Metrics: top-1 zero-shot accuracy top-5 accuracy weighted average accuracy 95% binomial confidence intervals
Baselines: OpenAI CLIP ViT-B/16 BioCLIP ViT-B/16 MetaCLIP-CC ViT-L/14 BioCLIP (iNat21)
Builds on: CLIP BioCLIP MetaCLIP TreeOfLife-10M
Closest papers by meaning
- BioCLIP: A Vision Foundation Model for the Tree of Life (CVPR 2024 · similarity 0.89)
- Open-Insect: Benchmarking Open-Set Recognition of Novel Species in Biodiversity Monitoring (NeurIPS 2025 · similarity 0.87)
- Insect-Foundation: A Foundation Model and Large-scale 1M Dataset for Visual Insect Understanding (CVPR 2024 · similarity 0.87)
- BIOSCAN-5M: A Multimodal Dataset for Insect Biodiversity (NeurIPS 2024 · similarity 0.86)
- BioCLIP 2: Emergent Properties from Scaling Hierarchical Contrastive Learning (NeurIPS 2025 · similarity 0.86)
- DivShift: Exploring Domain-Specific Distribution Shifts in Large-Scale, Volunteer-Collected Biodiversity Datasets (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.