Research map / Label-efficient and robust learning / Recognition and long-tail
CODA: A Correlation-Oriented Disentanglement and Augmentation Modeling Scheme for Better Resisting Subpopulation Shifts
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CODA, which disentangles spurious and causal latent codes with a bi-branch VAE-style encoder plus decoy classifier, then augments data by recombining latents and trains classifiers with a reweighted consistency loss to resist spurious correlations and group imbalance.
Problem
Generalization under subpopulation shifts combining spurious correlations and group imbalance (SC-GI), where models learn spurious attributes and underperform on minority groups.
Approach
A correlation-oriented disentanglement (COD) bi-branch encoder trained with a decoy classifier and reconstruction loss separates spurious (z) from invariant (t) latents; decoder recombination generates synthetic samples that vary spurious attributes while keeping class, and a reweighted consistency loss enforces consistent predictions on original and synthetic samples, plugging into ERM, RWG, or G
Key results
- On CelebA, CODA+GDRO raised worst-group accuracy to 89.26% from GDRO's 88.22%, and CODA+RWG reduced the max group accuracy gap to 7.28% from RWG's 10.78%.
- On ColoredMNIST, CODA+RWG achieved 72.11% worst-group accuracy versus 71.64% for RWG, with the max gap falling to 2.36%.
- On MultipleColoredMNIST (10 classes, 100 groups), CODA+ERM improved worst-group accuracy from 0% (ERM) to 91.85%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add CODA augmentation and reweighted consistency to ERM | ERM · ColoredMNIST | worst-group accuracy: 2.08 ± 0.70 -> 71.74 ± 0.24 | Helped |
| add CODA augmentation and reweighted consistency to RWG | RWG · ColoredMNIST | worst-group accuracy: 71.64 ± 0.15 -> 72.11 ± 0.51 | Helped |
| add CODA augmentation and reweighted consistency to GDRO | GDRO · CelebA | worst-group accuracy: 88.22 ± 0.89 -> 89.26 ± 0.26 | Helped |
| add CODA augmentation and reweighted consistency to GDRO | GDRO · CelebA | maximum group accuracy gap: 4.92 ± 1.11 -> 4.01 ± 0.24 | Helped |
| add CODA augmentation and reweighted consistency to ERM | ERM · CelebA | average accuracy: 95.05 ± 0.38 -> 91.89 ± 0.35 | Hurt |
| add synthetic samples by increasing L from 0 to 4 (ablation) | CODA+ERM · MultipleColoredMNIST | worst-group accuracy: 0.00 ± 0.00 -> 91.85 ± 0.68 | Helped |
| increase synthetic samples per instance from L=2 to L=4 (ablation) | CODA+RWG · MultipleColoredMNIST | worst-group accuracy: 91.35 ± 0.70 -> 91.10 ± 1.16 | No clear effect |
| use a 3-layer CNN instead of ResNet-18 encoders (ablation) | CODA+ERM · ColoredMNIST | worst-group accuracy: 71.74 ± 0.24 -> 67.50 ± 1.08 | Hurt |
Surprising findings
- Reconstruction quality correlates with final robust accuracy: a 3-CNN encoder with much higher reconstruction loss gave lower worst-group accuracy than ResNet-18 encoders.
- On CelebA, CODA+ERM and CODA+RWG had slightly lower average accuracy than the ERM baseline (91.89 vs 95.05 average for ERM), a trade-off for worst-group gains.
Limitations
- Requires pre-identified spurious attributes, which need costly labeling.
- Training is more resource-intensive due to the sample augmentation process.
- Augmentation quality depends on reconstruction quality; weaker encoders reduce robust accuracy.
Experimental setup
Datasets: ColoredMNIST CelebA MultipleColoredMNIST ColoredMNIST v2 ColoredMNIST v3 ColoredMNIST v4
Benchmarks: ColoredMNIST CelebA MultipleColoredMNIST ColoredMNIST v2 ColoredMNIST v3 ColoredMNIST v4
Metrics: worst-group accuracy average accuracy maximum group accuracy gap
Baselines: ERM RWG GDRO LfF JTT
Builds on: VAE beta-VAE Soft-IntroVAE group DRO reweighting groups
Closest papers by meaning
- Change is Hard: A Closer Look at Subpopulation Shift (ICML 2023 · similarity 0.86)
- Mitigating Spurious Correlation via Distributionally Robust Learning with Hierarchical Ambiguity Sets (ICLR 2026 · similarity 0.85)
- Group-robust Sample Reweighting for Subpopulation Shifts via Influence Functions (ICLR 2025 · similarity 0.85)
- Diverse Prototypical Ensembles Improve Robustness to Subpopulation Shift (ICML 2025 · similarity 0.83)
- Rectifying Soft-Label Entangled Bias in Long-Tailed Dataset Distillation (NeurIPS 2025 · similarity 0.83)
- $\alpha$TC-VAE: On the relationship between Disentanglement and Diversity (ICLR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.