Research map / Label-efficient and robust learning / OOD and anomaly detection
Diagnosing and Rectifying Fake OOD Invariance: A Restructured Causal Approach
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a ReStructured SCM (RS-SCM) and a conditional-mutual-information feature-selector (IIL) that rectifies fake invariant features left by InvRat/IIB, evaluated on two diagnostic CS-MNIST variants and DomainBed.
Problem
Invariant representation learning methods (InvRat, IIB, IRM) can recover "fake invariant" features that look domain-invariant in training environments but fail under unseen environments, as caused by spurious shortcut features.
Approach
Unifies PIIF and FIIF structural causal models into a RS-SCM that includes a fake-invariant variable Z_F = Phi(Z_s), then uses a neural soft-feature selector trained with a conditional mutual information objective (plus an MI maximization term against trivial solutions) alternately with InvRat/IIB to separate Z_c from Z_F.
Key results
- On CS-MNIST-COCO, IIB(+ours) reaches 69.59 OOD accuracy under Z/Z_F shift versus 66.38 for IIB; InvRat(+ours) reaches 66.42 versus 61.75 for InvRat.
- On DomainBed, IIB(+ours) averages 64.4 versus 63.4 for IIB, and InvRat(+ours) averages 62.9 versus 61.3 for InvRat.
- Removing the MI maximization term drops IIL performance noticeably, which the authors attribute to trivial CMI solutions.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add IIL rectification to IIB | IIB · CS-MNIST-CIFAR, Z/Z_F | accuracy: 69.42 -> 70.54 | Helped |
| add IIL rectification to IIB | IIB · CS-MNIST-CIFAR, Z/(Z_s,Z_F) | accuracy: 66.23 -> 68.37 | Helped |
| add IIL rectification to IIB | IIB · CS-MNIST-CIFAR, ID and Z/Z_s | accuracy: 96.76 -> 95.17; 70.98 -> 70.19 | Hurt |
| add IIL rectification to IIB | IIB · CS-MNIST-COCO, ID and OOD shifts | accuracy: 92.44 -> 93.17; 61.14 -> 63.72; 66.38 -> 69.59; 57.62 -> 62.35 | Helped |
| add IIL rectification to IIB | IIB · DomainBed average | accuracy: 63.4 -> 64.4 | Helped |
| add IIL rectification to IIB | IIB · VLCS | accuracy: 77.2 -> 77.6 | No clear effect |
| enable neural feature selector s(·) (ablation) | InvRat (+ours) · CS-MNIST-COCO, ID | accuracy: 90.34 -> 91.72 | Helped |
| enable neural feature selector s(·) (ablation) | InvRat (+ours) · CS-MNIST-COCO, Z/Z_F | accuracy: 62.46 -> 66.42 | Helped |
| add neural feature selector-trained IIL features, even with selector disabled at test time (ablation) | InvRat · CS-MNIST-COCO, Z/Z_F | accuracy: 61.75 -> 62.46 | Helped |
| include I(Z_c; h(X)/Z_c) maximization (ablation) | IIL with InvRat · CS-MNIST-COCO, Z/Z_F | accuracy: 64.26 -> 66.42 | Helped |
| include I(Z_c; h(X)/Z_c) maximization (ablation) | IIL with IIB · CS-MNIST-COCO, Z/Z_F | accuracy: 67.89 -> 69.59 | Helped |
| vary λ (ablation) | InvRat (+ours) · CS-MNIST-COCO, ID versus OOD Z/Z_F | accuracy | Mixed |
Surprising findings
- Under the combined spurious and fake-invariant shift, most baselines perform worse than under spurious shift alone, suggesting fake invariant shift is not eliminated by addressing spurious shift.
Limitations
- Gains on VLCS are very limited (+0.0 for InvRat, +0.4 for IIB).
- The fake invariance effect is argued to occur only when training environments are insufficient to cover all spurious factors.
- Diagnostic benchmarks are synthetic and built by the authors.
Experimental setup
Datasets: CS-MNIST CIFAR COCO VLCS PACS Office-Home Terra-Incognita DomainNet
Benchmarks: CS-MNIST-CIFAR CS-MNIST-COCO VLCS PACS Office-Home Terra-Incognita DomainNet
Metrics: accuracy
Baselines: ERM IRM IRM+IB InvRat IIB VREx CausalIRL
Builds on: InvRat IIB IRM MINE
Closest papers by meaning
- Invariant Causal Representation Learning for Out-of-Distribution Generalization (ICLR 2022 · similarity 0.91)
- Dissecting the Failure of Invariant Learning on Graphs (NeurIPS 2024 · similarity 0.89)
- Measure-Theoretic Anti-Causal Representation Learning (NeurIPS 2025 · similarity 0.86)
- Learning Linear Causal Representations from General Environments: Identifiability and Intrinsic Ambiguity (NeurIPS 2024 · similarity 0.86)
- From Bricks to Bridges: Product of Invariances to Enhance Latent Space Communication (ICLR 2024 · similarity 0.86)
- Causal Representation Learning Made Identifiable by Grouping of Observational Variables (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.