Research map / Label-efficient and robust learning / Recognition and long-tail
The Cells Out of Sample (COOS) dataset and benchmarks for measuring out-of-sample generalization of image classifiers
Alex Lu, Amy Lu, Wiebke Schormann, Marzyeh Ghassemi, David Andrews, Alan Moses
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Move from in-distribution Test1 to months-later Test3 | DeepLoc end-to-end CNN · COOS-7 | class-balanced error (%): 1.2 -> 7.4 | Hurt |
| Move from Test1 to different-microscope Test4 | DeepLoc end-to-end CNN · COOS-7 | class-balanced error (%): 1.2 -> 5.4 | Hurt |
| Move from Test1 to Test3 | Texture L1 LR · COOS-7 | class-balanced error (%): 6.8 -> 12.0 | Hurt |
| Move from Test1 to Test4 | Texture kNN · COOS-7 | class-balanced error (%): 11.8 -> 25.6 | Hurt |
| Use kNN on last-layer DeepLoc features instead of the CNN classifier | DeepLoc · COOS-7 Test3 | class-balanced error (%): 7.4 -> 6.9 | No clear effect |
| Replace kNN with L1 logistic regression on Texture features | Texture · COOS-7 Test3 | class-balanced error (%): 17.6 -> 12.0 | Improved |
| Replace kNN with L1 logistic regression on Texture features | Texture · COOS-7 Test4 | class-balanced error (%): 25.6 -> 12.1 | Improved |
| Replace kNN with L1 logistic regression on DeepLoc features | DeepLoc (FC2) · COOS-7 Test3 | class-balanced error (%): 6.9 -> 7.7 | No clear effect |
| Use random forest instead of kNN on VGG16 Conv4_2 features | VGG16 · COOS-7 Test1 | class-balanced error (%): 7.8 -> 7.4 | Improved |
| Use deeper PCI layer (Conv5 vs Conv3) with L1 LR | PCI · COOS-7 Test3 | class-balanced error (%): 9.2 -> 11.4 | Hurt |
| Use deeper PCI layer (Conv5 vs Conv3) with L1 LR | PCI · COOS-7 Test4 | class-balanced error (%): 7.4 -> 5.7 | Improved |
| Use VGG16 Conv4_2 instead of Conv3_3 features with L1 LR | VGG16 · COOS-7 Test3 | class-balanced error (%): 11.3 -> 8.0 | Improved |
Closest papers by meaning
- SELECT: A Large-Scale Benchmark of Data Curation Strategies for Image Classification (NeurIPS 2024 · similarity 0.84)
- Pooling Image Datasets with Multiple Covariate Shift and Imbalance (ICLR 2024 · similarity 0.84)
- MorphoHELM: A Comprehensive Benchmark for Evaluating Representations for Microscopy-Based Morphology Assays (NeurIPS 2026 · similarity 0.84)
- Benchmarking Out-of-Distribution Generalization Capabilities of DNN-based Encoding Models for the Ventral Visual Cortex. (NeurIPS 2024 · similarity 0.83)
- A Decade's Battle on Dataset Bias: Are We There Yet? (ICLR 2025 · similarity 0.83)
- ViTally Consistent: Scaling Biological Representation Learning for Cell Microscopy (ICML 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
