Research map / Label-efficient and robust learning / Domain adaptation and generalization
Towards Trustworthy Machine Learning Under Distribution Shifts
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
A PhD thesis on trustworthy machine learning under distribution shifts, covering OOD data harnessing (HOOD), sharpness-based DRO for photon-limited corruptions (SharpDRO), variant-parameter invariant learning (EVIL), machine vision therapy with MLLMs, and out-of-modal generalization.
Problem
Machine learning models degrade under distribution shifts (perturbation, domain, modality shift), and trustworthiness (robustness, explainability, adaptability) is needed alongside generalization.
Approach
Unifies several prior works from the author under a framework of three shift types and three trustworthiness aspects, with each chapter proposing a method for one shift scenario (e.g., causal content/style disentanglement with benign/malign OOD augmentation, worst-case sharpness minimization, MLLM in-context denoising for vision).
Key results
- HOOD achieves 86.12% AUROC on CIFAR10 vs. ID with LSUN OOD, versus 84.03% for OpenGAN (ID CIFAR10, LSUN).
- HOOD reaches 72.4% test accuracy on VisDA open-set DA, about 6 points above the second-best STA (66.4%).
- In open-set SSL on CIFAR10 with 400 labels per class, HOOD gets 50.74% corrupted accuracy versus 45.79% for OpenMatch.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add disentanglement loss (ablation) | HOOD · OOD detection (CIFAR10 ID, LSUN OOD) | AUROC: 84.94±1.3 -> 86.12±0.6 | Helped |
| Add benign OOD training data (ablation) | HOOD · OOD detection (CIFAR10 ID, LSUN OOD) | AUROC: 85.95±1.8 -> 86.12±0.6 | Helped |
| Add malign OOD training data (ablation) | HOOD · OOD detection (CIFAR10 ID, LSUN OOD) | AUROC: 82.50±2.2 -> 86.12±0.6 | Helped |
| Add both benign and malign OOD training augmentations (ablation) | HOOD · OOD detection (CIFAR10 ID, LSUN OOD) | AUROC: 80.83±0.8 -> 86.12±0.6 | Helped |
| Add disentanglement loss (ablation) | HOOD · Open-set SSL (CIFAR10, 400 labels/class) | accuracy: 82.55±1.1 -> 86.22±2.7 | Helped |
| Add benign OOD training data (ablation) | HOOD · Open-set SSL (CIFAR10, 400 labels/class) | accuracy: 83.32±2.0 -> 86.22±2.7 | Helped |
| Add malign OOD training data (ablation) | HOOD · Open-set SSL (CIFAR10, 400 labels/class) | accuracy: 85.40±0.8 -> 86.22±2.7 | Helped |
| Add both benign and malign OOD training augmentations (ablation) | HOOD · Open-set SSL (CIFAR10, 400 labels/class) | accuracy: 81.14±1.2 -> 86.22±2.7 | Helped |
| Add disentanglement loss (ablation) | HOOD · Open-set DA (VisDA) | accuracy: 64.6±0.9 -> 72.4±1.6 | Helped |
| Add benign OOD training data (ablation) | HOOD · Open-set DA (VisDA) | accuracy: 66.3±2.5 -> 72.4±1.6 | Helped |
| Add malign OOD training data (ablation) | HOOD · Open-set DA (VisDA) | accuracy: 71.8±1.2 -> 72.4±1.6 | Helped |
| Add both benign and malign OOD training augmentations (ablation) | HOOD · Open-set DA (VisDA) | accuracy: 65.4±1.2 -> 72.4±1.6 | Helped |
Surprising findings
- Malign OOD data is essential for OOD detection, while benign OOD data mainly helps open-set SSL and open-set DA.
- HOOD trails OpenMatch on clean accuracy in open-set SSL despite using fewer transformations, yet is more robust under corruption.
Limitations
- Training time increases due to causal disentanglement and adversarial training.
- HOOD slightly underperforms OpenMatch on clean accuracy in open-set SSL.
Experimental setup
Datasets: CIFAR10 CIFAR100 SVHN ImageNet Office VisDA LSUN DTD CUB Flowers Caltech Dogs
Benchmarks: CIFAR10 CIFAR100 CIFAR10-C CIFAR100-C Office VisDA
Metrics: AUROC clean accuracy corrupted accuracy test accuracy
Baselines: OpenGAN OpenMatch T2T OSBP UAN STA Likelihood Ratio ODIN GroupDRO
Builds on: OpenMatch distributionally robust optimization OpenGAN Machine Vision Therapy
Closest papers by meaning
- Uncovering the Connections Between Adversarial Transferability and Knowledge Transferability (ICML 2021 · similarity 0.85)
- Mutual Transfer Learning for Massive Data (ICML 2020 · similarity 0.84)
- A Mathematical Framework for Quantifying Transferability in Multi-source Transfer Learning (NeurIPS 2021 · similarity 0.83)
- On the Benefits of Public Representations for Private Transfer Learning under Distribution Shift (NeurIPS 2024 · similarity 0.83)
- Wasserstein Transfer Learning (NeurIPS 2025 · similarity 0.81)
- A High-Dimensional Statistical Method for Optimizing Transfer Quantities in Multi-Source Transfer Learning (NeurIPS 2025 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.