Research map / Label-efficient and robust learning / Recognition and long-tail
Agile Modeling: From Concept to Classifier in Minutes
Otilia Stretcu, Edward Vendrow, Kenji Hata, Krishnamurthy Viswanathan, Vittorio Ferrari, Sasan Tavakkol, Wenlei Zhou, Aditya Avinash, Emming Luo, Neil Gordon Alldrin, MohammadHossein Bateni, Gabriel Berger and 6 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train supervised model on user ratings instead of zero-shot scoring (a few more minutes of rating) | Agile Modeling · User study, 14 concepts | AUC PR: +3% after 5 minutes, +16% after 24 minutes | Improved |
| initial model trained on only 100 images vs zero-shot, CLIP embeddings | Agile Modeling · User study, 14 concepts | AUC PR | Hurt |
| label with the user instead of crowd raters on hard concepts (ablation) | Agile Modeling · Hard concepts, CLIP embeddings | AUC PR: User-100 above Crowd-500 | Improved |
| use margin plus positive mining instead of plain margin active learning (ablation) | Agile Modeling · User study, 14 concepts | AUC PR | No clear effect |
| use smaller active learning batch size (50 vs 100 vs 200) at fixed rating budget (ablation) | Agile Modeling · User study, 14 concepts | AUC PR | Improved |
| use ALIGN instead of CLIP embeddings (ablation) | Agile Modeling · User study, final model | AUPR: 0.68 -> 0.76 | Improved |
| use ALIGN instead of CLIP embeddings (ablation) | Agile Modeling · User study, final model | AUROC: 0.72 -> 0.80 | Improved |
| apply Agile Modeling with simulated users on 100 ImageNet21k concepts | Agile Modeling · ImageNet21k, 100 concepts | AUC PR: +20% over zero-shot | Improved |
Closest papers by meaning
- Modeling Collaborator: Enabling Subjective Vision Classification With Minimal Human Effort via LLM Tool-Use (CVPR 2024 · similarity 0.90)
- I-CEE: Tailoring Explanations of Image Classification Models to User Expertise (AAAI 2024 · similarity 0.84)
- V2C-CBM: Building Concept Bottlenecks with Vision-to-Concept Tokenizer (AAAI 2025 · similarity 0.84)
- Intriguing Properties of Generative Classifiers (ICLR 2024 · similarity 0.83)
- Classifier-to-Bias: Toward Unsupervised Automatic Bias Detection for Visual Classifiers (CVPR 2025 · similarity 0.83)
- Automated Detection of Visual Attribute Reliance with a Self-Reflective Agent (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
