Research map / Label-efficient and robust learning / OOD and anomaly detection
Rich Feature Construction for the Optimization-Generalization Dilemma
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Initialize OoD methods with a Bonsai feature representation instead of ERM pretraining | OoD methods on ColoredMNIST · ColoredMNIST | accuracy: 68.9$\pm$1.1 -> 66.5$\pm$1.5 (IRMv1); 69.3$\pm$0.7 -> 70.3$\pm$0.4 (vREx); 65.5$\pm$1.1 -> 69.8$\pm$0.6 (SD); 50.7$\pm$4 | Helped |
| Freeze the Bonsai representation rather than allow the OoD method to update it (ablation) | OoD methods on ColoredMNIST · ColoredMNIST | accuracy: 66.5$\pm$1.5 -> 69.9$\pm$0.6 (IRMv1); 70.3$\pm$0.4 -> 69.9$\pm$0.4 (vREx); 69.8$\pm$0.6 -> 70.4$\pm$0.4 (SD); 69.4$\pm$0 | Mixed |
| Initialize ERM with Bonsai features | ERM · ColoredMNIST | accuracy: 27.3$\pm$0.4 -> 43.4$\pm$2.8 | Helped |
| Use a frozen Bonsai representation rather than a second-easiest-feature strategy | PI · InverseColoredMNIST | accuracy: 51.0$\pm$4.7 (PI) versus 80.3$\pm$2.2 to 84.0$\pm$1.2 (frozen Bonsai OoD methods) | Helped |
| Initialize with a two-round Bonsai representation instead of ERM pretraining | ERM, IRMv1, vREx, and CLOvE · Camelyon17, OoD Tune | test accuracy: 70.2$\pm$8.7 -> 74.7$\pm$4.3 (ERM); 68.5$\pm$6.2 -> 75.3$\pm$4.8 (IRMv1); 69.1$\pm$13.2 -> 76.4$\pm$5.3 (vREx); 69.0$\pm | Helped |
| Freeze the two-round Bonsai representation rather than fine-tune it (ablation) | ERM, IRMv1, vREx, and CLOvE · Camelyon17, OoD Tune | test accuracy: 74.7$\pm$4.3 -> 78.6$\pm$2.6 (ERM); 75.3$\pm$4.8 -> 79.1$\pm$2.1 (IRMv1); 76.4$\pm$5.3 -> 79.5$\pm$2.7 (vREx); 76.6$\pm$ | Helped |
| Increase Bonsai discovery rounds from two to three (ablation) | Frozen Bonsai representations on Camelyon17 · Camelyon17, OoD Tune | test accuracy: 78.6$\pm$2.6 -> 73.3$\pm$5.3 (ERM); 79.1$\pm$2.1 -> 75.5$\pm$3.8 (IRMv1); 79.5$\pm$2.7 -> 75.1$\pm$5.3 (vREx); 78.6$\pm$ | Hurt |
| Use synthesis-phase L2 weight decay of 1e-2 rather than 1e-6 (ablation) | Frozen two-round Bonsai downstream models · Camelyon17, OoD Tune | test accuracy: 78.6$\pm$2.6 -> 75.5$\pm$7.4 (ERM); 79.1$\pm$2.1 -> 75.4$\pm$7.5 (IRMv1); 79.5$\pm$2.7 -> 75.8$\pm$7.3 (vREx) | Hurt |
| Use synthesis-phase L2 weight decay of 1e-4 rather than 1e-6 (ablation) | Frozen two-round Bonsai downstream models · Camelyon17, OoD Tune | test accuracy: 78.6$\pm$2.6 -> 78.8$\pm$2.3 (ERM); 79.1$\pm$2.1 -> 78.9$\pm$2.3 (IRMv1); 79.5$\pm$2.7 -> 79.7$\pm$1.7 (vREx) | No clear effect |
| Randomly initialize the top-layer classifier rather than average the discovery classifiers (ablation) | Downstream methods on two-round Bonsai · Camelyon17 | test accuracy: Table 5 reports comparable values; e.g. 2-Bonsai-cf vREx: 79.5$\pm$2.7 -> 79.7$\pm$2.4 (OoD Tune) | No clear effect |
| Combine features from different discovery rounds through the synthesis episode (ablation) | Bonsai representation · Camelyon17 | OoD test accuracy: 66.6$\pm$9.8; 73.2$\pm$5.7; 61.8$\pm$10.2 (individual rounds) versus 78.2$\pm$2.6 (ERM on frozen 2-Bonsai-cf) | Helped |
Closest papers by meaning
- OOD-Chameleon: Is Algorithm Selection for OOD Generalization Learnable? (ICML 2025 · similarity 0.86)
- Bridging OOD Detection and Generalization: A Graph-Theoretic View (NeurIPS 2024 · similarity 0.85)
- Making Classic GNNs Strong Baselines Across Varying Homophily: A Smoothness–Generalization Perspective (NeurIPS 2025 · similarity 0.85)
- Changing the Training Data Distribution to Reduce Simplicity Bias Improves In-distribution Generalization (NeurIPS 2024 · similarity 0.85)
- Get rich quick: exact solutions reveal how unbalanced initializations promote rapid feature learning (NeurIPS 2024 · similarity 0.84)
- Ensemble Pruning for Out-of-distribution Generalization (ICML 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.