AI Paper Atlas

Research map / Probabilistic and scientific modeling / Molecules and proteins

Identifying and Benchmarking Natural Out-of-Context Prediction Problems

David Madras, Richard Zemel

NeurIPS 2021 · Probabilistic and scientific modeling › Molecules and proteins · approach: Sparse concepts and hypothesis testing · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use environment-level rather than label-level reweightingResNet-50 classifiers · NOOCh-CE, average of 12 tasksAUC on hard examples: 0.870 -> 0.905Improved
Use environment-level rather than label-level reweightingResNet-50 classifiers · NOOCh-Gist, average of 12 tasksAUC on hard examples: 0.830 -> 0.849Improved
Use GDRO instead of ERMResNet-50 classifiers · WaterbirdsAUC-Hard: 0.691 -> 0.929Improved
Evaluate on gist-defined rather than CE-defined challenge setsrobust learning methods · NOOCh average of 12 tasksAUC on hard examples: 0.911 -> 0.849Hurt
Use environment-based information for trainingrobust learning methods · NOOCh-CE and NOOCh-GistAUC on hard examples: 0.865 -> 0.911; 0.826 -> 0.849Improved
Add data augmentation to ERM (ablation)ERM · NOOChperformanceNo clear effect
Add an auxiliary prediction head for the highest-alpha context variable (ablation)ERM · NOOChperformanceNo clear effect
Train ERM from scratch instead of using pretrained features (ablation)ERM · NOOChperformanceHurt
Add adaptive affine transformations after each ResNet block (ablation)ERM · NOOCh hard examplesperformanceImproved
Use logistic regression on pretrained ImageNet features instead of finetuned ERM (ablation)ResNet-50 pretrained features · NOOChperformanceHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy