Research map / Image generation / Text-to-image diffusion
Identification of Systematic Errors of Image Classifiers on Rare Subgroups
Jan Hendrik Metzen, Robin Hutmacher, N. Grace Hua, Valentyn Boreiko, Dan Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| condition image synthesis on the subgroup prompt instead of an unconditional class prompt (ablation) | PromptAttack · Car operational design domain (100 subgroups) | subgroup coverage (95% CI bound on p(z)): CI upper bound less than 0.0001 (unconditional) -> CI lower bound greater than 0.005 (conditional) | Improved |
| use Stable Diffusion v1.5 instead of v2-base or v2-1-base (ablation) | PromptAttack · Zero-shot systematic error benchmark | rank of ground-truth subgroup | Improved |
| increase samples per subgroup n_s to at least 16 (ablation) | PromptAttack · Zero-shot systematic error benchmark | rank of ground-truth subgroup | Improved |
| reduce diffusion inference steps to n_t=5 (ablation) | PromptAttack · Zero-shot systematic error benchmark | rank of ground-truth subgroup | No clear effect |
| raise class prompt weight w_c from 1.0 to 1.5 (ablation) | PromptAttack · Zero-shot systematic error benchmark | rank of ground-truth subgroup | Improved |
| increase class prompt weight beyond 1.5 (ablation) | PromptAttack · Zero-shot systematic error benchmark | rank of ground-truth subgroup | Hurt |
| use combinatorial testing with n_C=3 to test a subset of subgroups | PromptAttack vehicle experiment · ImageNet minivan | subgroups tested: 1.230 of 18.720 subgroups explored | Improved |
| shift all semantic dimensions jointly rather than a single dimension | ViT-B/16 on minivan prompts · ImageNet minivan to snowplow | samples misclassified as snowplow: 300 of 1000 samples misclassified | Improved |
Closest papers by meaning
- Discover and Mitigate Multiple Biased Subgroups in Image Classifiers (CVPR 2024 · similarity 0.90)
- Subgroups Matter for Robust Bias Mitigation (ICML 2025 · similarity 0.88)
- Project-Probe-Aggregate: Efficient Fine-Tuning for Group Robustness (CVPR 2025 · similarity 0.85)
- Doomed to Re-Annotate, Forever: The ImageNet Story (NeurIPS 2026 · similarity 0.84)
- Who Evaluates the Evaluations? Objectively Scoring Text-to-Image Prompt Coherence Metrics with T2IScoreScore (TS2) (NeurIPS 2024 · similarity 0.84)
- Improving Subgroup Robustness via Data Selection (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
