Research map / Vision-language and multimodal / Cross-modal learning
BIOSCAN-5M: A Multimodal Dataset for Insect Biodiversity
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces BIOSCAN-5M, a multimodal insect/arthropod dataset of over 5 million specimens (images, DNA barcodes, taxonomy, geography, size), and benchmarks DNA language modeling, zero-shot clustering, and contrastive multimodal retrieval on it.
Problem
Insect biodiversity monitoring needs large, multimodal, taxonomically annotated reference data; existing biological image datasets lack DNA barcodes, geography, and fine-grained labels, and benchmarks rarely exploit multiple modalities.
Approach
A 5M-specimen dataset pairing high-resolution images, raw COI DNA barcodes, BINs, taxonomy at seven ranks, and geographic/size metadata, with seen/unseen/heldout species partitions for closed- and open-world evaluation, plus three benchmarks that test each modality and their combination.
Key results
- Own 8-4-4 barcode model reaches 99.28% fine-tuned species accuracy on seen species, slightly above DNABERT-2 (99.23%).
- On unseen-species genus 1NN probing, the own model achieves 47.03% versus 23.15% for BarcodeBERT.
- Zero-shot clustering: DNA encoders outperform image encoders across ranks (species AMI 91% for best DNA model vs 21% image agreement at species level); CLIBD trained on 5M records improves over 1M records at most taxa.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Pretrain a BarcodeBERT-style transformer on BIOSCAN-5M DNA barcodes | DNA-based taxonomic classification · BIOSCAN-5M test_unseen | 1NN-probe accuracy: 29.88 -> 47.03 | Helped |
| Use BIOSCAN-5M-pretrained embeddings instead of DNABERT-S embeddings | DNA-based taxonomic classification · BIOSCAN-5M seen test | linear probe accuracy: 95.50 -> 94.47 | Hurt |
| Increase contrastive training data from 1M to 5M records | CLIBD multimodal retrieval · BIOSCAN-5M test, image-to-image, genus | macro top-1 accuracy, H.M.: 76.1 -> 84.4 | Helped |
| Add text alignment to the 5M image+DNA model (ablation) | CLIBD multimodal retrieval · BIOSCAN-5M test, DNA-to-DNA, family | macro top-1 accuracy, H.M.: 99.7 -> 99.7 | No clear effect |
| Add text alignment to the 5M image+DNA model (ablation) | CLIBD multimodal retrieval · BIOSCAN-5M test, image-to-image, species | macro top-1 accuracy, H.M.: 67.5 -> 67.1 | Hurt |
| Add image features to DNA embeddings for zero-shot clustering (ablation) | Cross-modal zero-shot clustering · BIOSCAN-5M test and test_unseen | AMI | Hurt |
| Reduce to one sample per barcode (ablation) | HyenaDNA zero-shot clustering · BIOSCAN-5M test and test_unseen, species | AMI: 90% -> 75% | Hurt |
| Use BIOSCAN-5M DNA-pretrained model instead of BarcodeBERT | DNA-based taxonomic classification · BIOSCAN-5M seen test | fine-tuned accuracy: 98.52 -> 99.28 | Helped |
| Use BIOSCAN-5M DNA-pretrained model instead of BarcodeBERT | DNA-based taxonomic classification · BIOSCAN-5M seen test | linear probe accuracy: 91.93 -> 94.47 | Helped |
| Increase DNA transformer tokenization length and vary architecture to 8-4-4 (ablation) | BIOSCAN-5M DNA encoder · BIOSCAN-5M validation | fine-tuned class-balanced accuracy: 93.8 -> 98.9 | Mixed |
Surprising findings
- Adding image features to DNA embeddings in zero-shot clustering decreased performance relative to DNA alone.
- Including the text modality in contrastive training slightly lowered species-level retrieval accuracy.
Limitations
- Sparse species labels (only about 9% of records have species labels) limit fine-grained species-level learning, especially for multimodal models.
- Sampling bias from collection locations and methods.
- Linear probe and 1NN comparisons use models with different embedding dimensions, a confound acknowledged by the authors.
Experimental setup
Datasets: BIOSCAN-5M BIOSCAN-1M ImageNet-1K ImageNet-21k BOLD
Benchmarks: BIOSCAN-5M
Metrics: accuracy linear probe accuracy 1NN-probe accuracy adjusted mutual information top-1 macro accuracy top-1 micro accuracy harmonic mean
Baselines: BarcodeBERT DNABERT-2 DNABERT-S Nucleotide Transformer HyenaDNA CNN baseline BLAST ResNet-50 ViT-B CLIP
Builds on: BarcodeBERT CLIBD BIOSCAN-1M CLIP DNABERT-2
Closest papers by meaning
- CLIBD: Bridging Vision and Genomics for Biodiversity Monitoring at Scale (ICLR 2025 · similarity 0.89)
- Open-Insect: Benchmarking Open-Set Recognition of Novel Species in Biodiversity Monitoring (NeurIPS 2025 · similarity 0.89)
- Insect-Foundation: A Foundation Model and Large-scale 1M Dataset for Visual Insect Understanding (CVPR 2024 · similarity 0.88)
- BioTrove: A Large Curated Image Dataset Enabling AI for Biodiversity (NeurIPS 2024 · similarity 0.86)
- BioCLIP: A Vision Foundation Model for the Tree of Life (CVPR 2024 · similarity 0.85)
- Noisy Ostracods: A Fine-Grained, Imbalanced Real-World Dataset for Benchmarking Robust Machine Learning and Label Correction Methods (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.