Research map / Label-efficient and robust learning / Recognition and long-tail
DivShift: Exploring Domain-Specific Distribution Shifts in Large-Scale, Volunteer-Collected Biodiversity Datasets
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces the DivShift framework and DivShift-NAWC, a ~7.3M-image iNaturalist plant dataset partitioned by five expert-verified biases, to measure how volunteer-data biases shift species recognition performance.
Problem
Volunteer-collected biodiversity datasets (e.g., iNaturalist) contain spatial, temporal, taxonomic, observer, and sociopolitical biases, but their effect on fine-grained species recognition performance is poorly understood.
Approach
Casts domain-specific biases as distribution shifts between data partitions and compares the drop in model accuracy across partitions to the Jensen-Shannon distance between label distributions, with a "strong/weak generalizer" analysis of train-partition choice.
Key results
- Bias-induced performance drops are smaller than the underlying label distribution shift (JSD) for all five bias partitions, classified as weakly biased overall.
- Training on engaged observers yields substantially better out-of-distribution performance than training on casual observers, with the performance gap attributed to image quality rather than data volume.
- A ViT-base model improved Top-1 per-image accuracy over ResNet-18 by roughly 15-20% depending on metric on the casual partition, while ResNet50 gave ~2-12% gains.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the Random partition rather than the iNat2021 Mini partition | ResNet-18 species recognition · DivShift-NAWC | Top1-Img accuracy: 33.7 -> 70.6 | Helped |
| Use ResNet50 instead of ResNet-18 (ablation) | Casual-observer training · DivShift-NAWC | Top1-Img accuracy: 71.1; -22.7 | Helped |
| Use ViT instead of ResNet-18 (ablation) | Casual-observer training · DivShift-NAWC | Top1-Img accuracy: 79.0; -23.1 | Helped |
| Train on Modified observations rather than Wild observations | ResNet-18 transfer to Wild test data · DivShift-NAWC spatial bias | Top1-Img difference: -35.3 -> -11.3 | Helped |
| Train outside the City Nature Challenge rather than on CNC observations | ResNet-18 transfer to CNC test data · DivShift-NAWC temporal bias | Top1-Img difference: -17.9 -> +1.9 | Helped |
| Train outside CNC rather than on CNC observations | ResNet-18 transfer to non-CNC test data · DivShift-NAWC temporal bias | Top1-Img difference: -17.9 | Helped |
| Train on Engaged rather than Casual observers' observations | ResNet-18 transfer to Casual test data · DivShift-NAWC observer bias | Top1-Img difference: -23.1 -> +4.9 | Helped |
| Balance training examples across species rather than retain the Long-tail distribution | ResNet-18 species recognition · DivShift-NAWC, rare species | Top1-RAR accuracy: 24.7 -> 26.6 | Helped |
| Keep maximal Long-tail training data rather than subsample frequent classes | ResNet-18 species recognition · DivShift-NAWC, frequent species | Top1-FAR accuracy: 26.6 -> 24.7 | Helped |
| Train on Casual rather than Engaged observers' observations | ResNet-18 transfer to Casual test data · DivShift-NAWC observer bias | Top1-Img difference: +4.9 | Hurt |
| Train on Wild rather than Modified observations | ResNet-18 transfer to Modified test data · DivShift-NAWC spatial bias | Top1-Img difference: -35.3 | Hurt |
| Train on CNC rather than non-CNC observations | ResNet-18 transfer to non-CNC test data · DivShift-NAWC temporal bias | Top1-Img difference: -17.9 | Hurt |
Surprising findings
- Biases confound model performance less than expected from the underlying label distribution shift.
- Training on observations from the City Nature Challenge alone generalizes worse than training on non-challenge observations, and wilderness-trained models overfit even on rare species.
Limitations
- No causal attribution of performance changes to specific biases.
- Only common supervised training approaches evaluated; no unsupervised or long-tailed learning methods.
- Per-image rather than per-observation train/test splitting may inflate in-distribution test performance.
Experimental setup
Datasets: DivShift-NAWC iNaturalist Open Data iNat2021 ImageNet
Benchmarks: DivShift-NAWC
Metrics: Top-1 per-image accuracy Top-1 per-species accuracy rarity-weighted Top-1 accuracy frequent/common/rare species accuracy land use category accuracy ecoregion accuracy Jensen-Shannon distance
Baselines: iNat2021 iNat2021 mini ImageNet Spatial stratified split Random split
Builds on: iNat2021 ImageNet
Closest papers by meaning
- Open-Insect: Benchmarking Open-Set Recognition of Novel Species in Biodiversity Monitoring (NeurIPS 2025 · similarity 0.87)
- BioTrove: A Large Curated Image Dataset Enabling AI for Biodiversity (NeurIPS 2024 · similarity 0.86)
- Spatial Clustering of Citizen Science Data Improves Downstream Species Distribution Models (AAAI 2025 · similarity 0.85)
- Combining Observational Data and Language for Species Range Estimation (NeurIPS 2024 · similarity 0.85)
- Feedforward Few-shot Species Range Estimation (ICML 2025 · similarity 0.85)
- GeoPlant: Spatial Plant Species Prediction Dataset (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.