Research map / Label-efficient and robust learning / Recognition and long-tail
Compositional Risk Minimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Compositional Risk Minimization (CRM), which trains an additive energy classifier on train groups and then extrapolates biases and adjusts priors to generalize to unseen attribute combinations under compositional shift.
Problem
Discriminative classifiers fail when some attribute combinations (groups) are entirely absent from training but appear at test time, i.e., compositional shift in multi-attribute classification.
Approach
Model p(z|x) with an additive energy classifier whose log-prior and learned per-group bias can be adjusted at test time; a theoretically derived extrapolated bias B* (computed from training data only) lets the classifier output correct probabilities for unseen groups inside the discrete affine hull of training groups.
Key results
- On Waterbirds averaged over dropped-group scenarios, CRM reaches 78.7% WGA versus 77.3% for sLA (best baseline), and in the worst-case drop scenario CRM gets 73.0% vs 69.0% for LC.
- On CelebA, CRM achieves 81.8% average WGA versus 67.8% for G-DRO, the strongest baseline.
- Ablation: using the learned bias instead of extrapolated bias drops Waterbirds WGA from 78.7% to 55.7%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use extrapolated bias instead of learned bias (ablation) | CRM · Waterbirds compositional shift | worst-group accuracy: 55.7 -> 78.7 | Helped |
| use extrapolated bias instead of learned bias (ablation) | CRM · CelebA compositional shift | worst-group accuracy: 58.9 -> 81.8 | Helped |
| use extrapolated bias instead of learned bias (ablation) | CRM · MetaShift compositional shift | worst-group accuracy: 58.7 -> 73.4 | Helped |
| use extrapolated bias instead of learned bias (ablation) | CRM · MultiNLI compositional shift | worst-group accuracy: 30.4 -> 58.7 | Helped |
| use extrapolated bias instead of learned bias (ablation) | CRM · CivilComments compositional shift | worst-group accuracy: 52.4 -> 67.9 | Helped |
| use extrapolated bias instead of learned bias (ablation) | CRM · NICO++ compositional shift | worst-group accuracy: 31.0 -> 40.3 | Helped |
| use CRM instead of sLA | Waterbirds · Waterbirds compositional shift | worst-group accuracy: 77.3 -> 78.7 | Helped |
| use CRM instead of G-DRO | CelebA · CelebA compositional shift | worst-group accuracy: 67.8 -> 81.8 | Helped |
| use CRM instead of LC | MetaShift · MetaShift compositional shift | worst-group accuracy: 68.2 -> 73.4 | Helped |
| use CRM instead of sLA | MultiNLI · MultiNLI compositional shift | worst-group accuracy: 55.0 -> 58.7 | Helped |
| use CRM instead of LC | CivilComments · CivilComments compositional shift | worst-group accuracy: 65.7 -> 67.9 | Helped |
| use CRM instead of G-DRO | NICO++ · NICO++ compositional shift | worst-group accuracy: 36.7 -> 40.3 | Helped |
Surprising findings
- Logit-adjustment baselines appear competitive on average but fail badly in the worst-case group-drop scenario.
- The learned bias alone (without extrapolation) gives large WGA drops, showing prior adjustment alone is insufficient.
Limitations
- Requires labeled attributes (group labels), which may not be available.
- Only a tractable discriminative approach; the generative extension is left open.
- Requires knowledge or assumption of the test prior (uniform by default).
Experimental setup
Datasets: Waterbirds CelebA MetaShift MultiNLI CivilComments NICO++
Benchmarks: Waterbirds CelebA MetaShift MultiNLI CivilComments NICO++
Metrics: average accuracy worst-group accuracy group-balanced accuracy
Baselines: ERM GroupDRO Logit Correction sLA IRM VREx Mixup
Builds on: Logit Correction supervised logit adjustment energy-based compositional generation independent mechanisms principle
Closest papers by meaning
- On the Statistical Mechanisms of Distributional Compositional Generalization (ICML 2025 · similarity 0.86)
- Does Data Scaling Lead to Visual Compositional Generalization? (ICML 2025 · similarity 0.86)
- When does compositional structure yield compositional generalization? A kernel theory. (ICLR 2025 · similarity 0.86)
- Scalable Evaluation and Neural Models for Compositional Generalization (NeurIPS 2025 · similarity 0.86)
- Generalizable Reasoning through Compositional Energy Minimization (NeurIPS 2025 · similarity 0.85)
- Transformation-Invariant Learning and Theoretical Guarantees for OOD Generalization (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.