Research map / Label-efficient and robust learning / Recognition and long-tail
Open-Insect: Benchmarking Open-Set Recognition of Novel Species in Biodiversity Monitoring
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Open-Insect, a large-scale fine-grained moth/insect dataset for open-set recognition across three regions, benchmarks 38 OSR methods, and proposes two auxiliary-data baselines (NovelBranch, Extended).
Problem
Open-set recognition for species identification in biodiversity monitoring is poorly evaluated on fine-grained, highly diverse taxa like insects; existing OSR benchmarks are small or use well-studied vertebrates and generic object categories.
Approach
A regional, geographically-grounded open-set benchmark for moths in which local open-set species come from neighboring regions and non-local (Australia) and non-moth open-sets test semantic shift, with matched auxiliary data per region; plus two simple auxiliary-data baselines using an extra classification head.
Key results
- MSP-based post-hoc methods (TempScale, RP-MSP) remain a strong baseline, with the best post-hoc methods exceeding 85% AUROC on all regions.
- Auxiliary-data methods improve C-America local open-set AUROC; Energy reaches 90.0 on local and 93.8 on non-local, versus 87.6 and 89.5 for the best regularization method LogitNorm.
- Replacing the realistic auxiliary set with TinyImageNet lowers C-America local AUROC by 4.82 pp and non-local by 5.54 pp.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use realistic Open-Insect auxiliary data instead of TinyImageNet (ablation) | Energy on C-America · C-America local moth | AUROC: 85.25 -> 90.07 | Helped |
| Use realistic Open-Insect auxiliary data instead of TinyImageNet (ablation) | Energy on C-America · C-America non-local moth | AUROC: 88.16 -> 93.7 | Helped |
| Increase the number of auxiliary species while fixing total image count (ablation) | NovelBranch with ImageNet-1K weights · C-America local moth | AUROC | Helped |
| Increase the number of auxiliary species while fixing total image count (ablation) | NovelBranch with ImageNet-1K weights · C-America non-local moth | AUROC | No clear effect |
| Initialize with ImageNet-1K pretrained weights instead of training from scratch | C-America classifiers · Open-Insect C-America local and non-local moth | AUROC: 94.04 to 95.38 | Helped |
| Use BioCLIP weights instead of ImageNet-1K weights | ViT-B-16 · Open-Insect subset | AUROC: 90.22 vs 85.02 | Helped |
| Use training-time regularization rather than a basic classifier | Regions with more closed-set data · NE-America and W-Europe local and non-local moth | AUROC | No clear effect |
| Use training-time regularization rather than a basic classifier | Regions with more closed-set data · NE-America and W-Europe non-moth | AUROC | Helped |
| Mask the moth, leaving only background and silhouette cues (ablation) | MSP on C-America images · C-America test subset | AUROC: 80.01 -> 46.57 | Hurt |
| Mask the background (ablation) | MSP on C-America images · C-America test subset | AUROC: 80.01 -> 76.01 | Hurt |
| Use local open-set species instead of non-local or non-moth open-set species | Open-Insect OSR methods · Three Open-Insect regions | AUROC | Hurt |
| Use simple MSP-based post-hoc scoring instead of more complex post-hoc approaches | Open-Insect OSR · Open-Insect | AUROC | No clear effect |
Surprising findings
- Simple post-hoc MSP-based methods remain competitive with more complex OSR and training-based methods on fine-grained moth data.
- Training-time regularization gives inconsistent gains; it does not improve local or non-local OSR for regions with large closed sets.
Limitations
- Only high-resolution images evaluated; no low-resolution camera-trap images included.
- Training-regularization and auxiliary-data methods trained once without multiple runs due to compute limits.
- Local open-set species are simulated from described species rather than truly novel species; BCI set is small (197 images).
Experimental setup
Datasets: AMI-GBIF GBIF moth images TinyImageNet ImageNet-1K
Benchmarks: Open-Insect
Metrics: AUROC closed-set accuracy AUOSCR OpenAUC OOSA TPR@5
Baselines: MSP OpenMax TempScale ODIN EBO MLS ReAct KNN LogitNorm ARPL OE MixOE
Builds on: OpenOOD OpenMax Outlier Exposure BioCLIP
Closest papers by meaning
- BIOSCAN-5M: A Multimodal Dataset for Insect Biodiversity (NeurIPS 2024 · similarity 0.89)
- Insect-Foundation: A Foundation Model and Large-scale 1M Dataset for Visual Insect Understanding (CVPR 2024 · similarity 0.88)
- BioTrove: A Large Curated Image Dataset Enabling AI for Biodiversity (NeurIPS 2024 · similarity 0.87)
- DivShift: Exploring Domain-Specific Distribution Shifts in Large-Scale, Volunteer-Collected Biodiversity Datasets (AAAI 2025 · similarity 0.87)
- CLIBD: Bridging Vision and Genomics for Biodiversity Monitoring at Scale (ICLR 2025 · similarity 0.85)
- Feedforward Few-shot Species Range Estimation (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.