Research map / Label-efficient and robust learning / OOD and anomaly detection

Perturb and Correct: Post-Hoc Ensembles using Affine Redundancy

NeurIPS 2026 · Label-efficient and robust learning › OOD and anomaly detection · approach: Uncertainty and misclassification detection · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Perturb-and-Correct (P&C), a post-hoc method that builds an epistemic ensemble from one pretrained network by randomly perturbing a hidden layer and refitting the next affine layer by ridge least squares on ID calibration data, with theory on the post-correction residual and experiments on MuJoCo, CIFAR-10, and ImageNet ViT-B/16.

Problem

Single trained networks may be indistinguishable in-distribution yet behave differently under shift; cheap post-hoc ensembles that expose this unresolved variation without the cost of training multiple models are needed for epistemic uncertainty and OOD detection.

Approach

Randomly perturb a low-dimensional hidden-layer parameter subspace, then refit the following affine layer by ridge regression to reproduce the base model's next-layer pre-activations on ID calibration data; the residual that survives correction (the transfer defect) produces ensemble disagreement away from calibration support.

Key results

• On MuJoCo across 11 environments and 10 seeds, P&C has the best mean rank on Far NLL (1.55) and Far AUROC (1.27), winning 9/11 on AUROC.
• On CIFAR-10 OpenOOD v1.5, P&C reaches Near AUROC 91.55 and Far AUROC 95.09 with Far FPR95 18.15, versus Deep Ensemble 94.63 Far AUROC and SCOD 92.56.
• On ImageNet-1K ViT-B/16, corrected P&C improves Near/Far AUROC to 76.49/87.89 over uncorrected perturbation at 72.53/84.40; Mahalanobis remains stronger at 78.82/92.55.

Surprising findings

• Mahalanobis distance on the final CLS representation outperforms P&C on pure ImageNet OOD ranking.
• The derivative-at-zero (linearized) approximation is only accurate at perturbation norms 30-300x smaller than those used, so the exact residual is needed.

Limitations

• Depends on calibration set covering intended ID behavior and well-conditioned correction design; perturbations must stay in ID-stable range.
• Member-dependent inference remains O(M); construction cost advantage over Deep Ensembles, not inference cost.
• Humanoid results are poor (Far-AUROC 0.343) and ID error is mixed across MuJoCo environments.

Experimental setup

Datasets: MuJoCo (Ant-v5, HalfCheetah-v5, Hopper-v5, and others) CIFAR-10 CIFAR-100 TinyImageNet MNIST SVHN Textures Places365 ImageNet-1K iNaturalist OpenImage-O SSB-hard

Benchmarks: MuJoCo dynamics prediction (11 environments) OpenOOD v1.5 CIFAR-10 ImageNet-1K OOD detection Banking77

Metrics: ID RMSE Far NLL AUROC FPR95 Spearman correlation top-1 accuracy NLL ECE

Baselines: Deep Ensemble MC Dropout SWAG Laplace Subspace Inference SCOD Mahalanobis ReAct+Energy LLLA Epinet MSP Uncorrected perturbation

Builds on: Deep Ensembles Subspace Inference SWAG Laplace approximation SCOD

Closest papers by meaning

• PEP: Parameter Ensembling by Perturbation (NeurIPS 2020 · similarity 0.86)
• Post-Hoc Uncertainty Calibration for Domain Drift Scenarios (CVPR 2021 · similarity 0.86)
• Progressive Ensemble Distillation: Building Ensembles for Efficient Inference (NeurIPS 2023 · similarity 0.85)
• DomED: Redesigning Ensemble Distillation for Domain Generalization (ICML 2026 · similarity 0.85)
• Post-Hoc Reversal: Are We Selecting Models Prematurely? (NeurIPS 2024 · similarity 0.84)
• Improving Perturbation-based Explanations by Understanding the Role of Uncertainty Calibration (NeurIPS 2025 · similarity 0.84)