Research map / Label-efficient and robust learning / Recognition and long-tail

PHOEBI: An Open-World Benchmark for Bacterial Identification in Phase-Contrast Microscopy

NeurIPS 2026 · Label-efficient and robust learning › Recognition and long-tail · approach: Partial and multi-label learning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces PHOEBI, a 120K-image wet-lab phase-contrast benchmark of 40 six-species bacterial mixtures with a leave-combinations-out protocol, showing gradient-trained per-image classifiers collapse on unseen mixtures while frozen-DINOv2 anchor-based decoders stay stable and support open-set rejection and novel-class discovery.

Problem

Multi-label identification of bacterial species in phase-contrast microscopy of polymicrobial samples, including species never seen in training, lacks a benchmark; standard classifiers fail to generalize to unseen species combinations.

Approach

Treat each tile of a spatially homogeneous slide image as an i.i.d. sample of the image label, and read each species' presence against fixed geometric prototypes (anchor-based decoders) over frozen DINOv2 tile features, avoiding per-combination decision surfaces that overfit.

Key results

• Gradient-trained fine-tuned backbones fall to 0.44-0.61 held-out F1 from 0.97-1.00 in-distribution, while attention MIL on frozen DINOv2 drops to 0.574 held-out F1 from 0.906.
• ProtoMatch (closed-form anchor decoder) reaches 0.683 held-out F1 versus 0.654 for the all-present constant predictor, with held-out macro F1 0.722.
• Sinkhorn-Knopp K=1 novel-class discovery reaches cluster accuracy 0.502 with known-class F1 drift of only -0.031, versus -0.377 for greedy cosine clustering.

Surprising findings

• Held-out combinations give even a constant all-present predictor a higher F1 than trained classifiers, so trained aggregators fall below a trivial reference.
• Training the unanchored ChannelGroup decoder lowers held-out F1 while raising validation F1, whereas training the simplex decoder leaves held-out F1 unchanged.

Limitations

• Acquired on a single microscope instrument with one operator.
• Only six rod-shaped species; taxonomic breadth limited.
• Open-set and discovery evaluated with one unseen species at a time and require pure-culture samples for anchor initialization.

Experimental setup

Datasets: PHOEBI

Benchmarks: PHOEBI

Metrics: per-sample F1 macro F1 exact match AUROC cluster accuracy delta F1

Baselines: ResNet-50 ConvNeXt-B ViT-B/16 IN21k DINOv3 ViT-S/16 CLIP ViT-B/16 SigLIP ViT-B/16 EVA-02 CLIP B/16 Florence-2 DaViT-B Attention MIL Prov-GigaPath

Builds on: DINOv2 sparsemax Attention MIL Sinkhorn-Knopp UNO

Closest papers by meaning

• BioCLIP: A Vision Foundation Model for the Tree of Life (CVPR 2024 · similarity 0.83)
• MorphoHELM: A Comprehensive Benchmark for Evaluating Representations for Microscopy-Based Morphology Assays (NeurIPS 2026 · similarity 0.82)
• BioTrove: A Large Curated Image Dataset Enabling AI for Biodiversity (NeurIPS 2024 · similarity 0.81)
• Micro-Bench: A Microscopy Benchmark for Vision-Language Understanding (NeurIPS 2024 · similarity 0.81)
• CLIBD: Bridging Vision and Genomics for Biodiversity Monitoring at Scale (ICLR 2025 · similarity 0.81)
• Fish-Vista: A Multi-Purpose Dataset for Understanding & Identification of Traits from Images (CVPR 2025 · similarity 0.81)