Research map / Label-efficient and robust learning / Recognition and long-tail
Consensus-Driven Active Model Selection
Justin Kay, Grant Van Horn, Subhransu Maji, Daniel Sheldon, Sara Beery
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use CODA instead of random sampling | active model selection · CoLA | cumulative regret at step 100: 368.1 -> 2226.7 | Hurt |
| use CODA instead of random sampling | active model selection · CivilComments | cumulative regret at step 100: 140.9 -> 318.6 | Hurt |
| use CODA instead of ModelSelector | active model selection · DomainNet126 painting→real | cumulative regret at step 100: 293.4 -> 2.4 | Improved |
| use CODA instead of ModelSelector | active model selection · DomainNet126 sketch→clipart | cumulative regret at step 100: 38.1 -> 432.4 | Hurt |
| add consensus-informed priors to CODA (ablation) | CODA · benchmark tasks | cumulative regret | Improved |
| add diagonal-weighted prior (vs uniform) to CODA (ablation) | CODA · benchmark tasks | cumulative regret | Mixed |
| use expected information gain acquisition instead of random or uncertainty sampling (ablation) | CODA · benchmark tasks | cumulative regret | Improved |
| use uncertainty sampling acquisition inside CODA instead of EIG (ablation) | CODA · benchmark tasks | cumulative regret | Mixed |
| use CODA consensus priors only (unsupervised) instead of EnsV | unsupervised model selection · 26 tasks | regret at step 0 | Mixed |
| use CODA instead of EnsV | unsupervised model selection · DomainNet126 sketch→real | regret at step 0: 4.5 -> 0.1 | Improved |
| use CODA instead of EnsV | unsupervised model selection · WILDS iwildcam | regret at step 0: 0.9 -> 6.1 | Hurt |
Closest papers by meaning
- Contextual Active Model Selection (NeurIPS 2024 · similarity 0.87)
- Active Statistical Inference (ICML 2024 · similarity 0.86)
- Scaling Up Active Testing to Large Language Models (NeurIPS 2025 · similarity 0.85)
- Prediction-Powered Active Testing (NeurIPS 2026 · similarity 0.84)
- What Does It Take to Build a Performant Selective Classifier? (NeurIPS 2025 · similarity 0.84)
- Robust Sampling for Active Statistical Inference (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
