Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Measuring Stochastic Data Complexity with Boltzmann Influence Functions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes IF-COMP, a temperature-scaled Boltzmann influence function approximation of the pNML distribution that yields calibrated predictions and stochastic complexity scores for labelled and unlabelled points.
Problem
Estimating predictive uncertainty and stochastic data complexity via the pNML distribution is computationally intractable and ill-posed (infinity problem) for overparameterized neural networks, which can fit arbitrary labels.
Approach
Temperature-scale the codelength and proximal Bregman objective to form a Boltzmann influence function (BIF), linearize the model in log-probability space so hindsight-optimal output probabilities are approximated with one EKFAC-based influence computation per label, enabling cheap pNML estimates.
Key results
- On CIFAR-10 mislabel detection with human noise, IF-COMP reaches 96.86 AUROC vs 95.38 for Self-IF and 90.50 for Trac-IN.
- IF-COMP achieves best OOD AUROC on MNIST (99.40 near / 99.97 far) and CIFAR-10 (92.23 near / 95.63 far) among OpenOOD baselines.
- IF-COMP gives 7.34x speedup over ACNML-EKFAC on MNIST LeNet and 12.4-14.71x on ResNet18 models.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use IF-COMP instead of Self-IF for CIFAR-10 mislabel detection | ResNet-18 · CIFAR-10 human noise | AUROC: 95.38 -> 96.86 | Helped |
| Use IF-COMP instead of other reported methods for CIFAR-10 mislabel detection | ResNet-18 · CIFAR-10 data-dependent, asymmetric 0.3, and symmetric 0.6 noise | AUROC: 55.77 -> 88.07; 87.30 -> 95.69; 64.40 -> 97.83 | Helped |
| Use IF-COMP instead of the strongest listed baseline | ResNet-18 · CIFAR-100 asymmetric 0.3 and symmetric 0.6 noise | AUROC: 72.15 -> 79.39; 58.40 -> 95.21 | Helped |
| Use IF-COMP instead of MLS or RMDS | LeNet or ResNet-18 · MNIST Near-OOD and Far-OOD | AUROC: 98.00 -> 99.40; 99.75 -> 99.97 | Helped |
| Use IF-COMP instead of KNN or VIM | ResNet-18 · CIFAR-10 Near-OOD and Far-OOD | AUROC: 90.64 -> 92.23; 93.48 -> 95.63 | Helped |
| Use IF-COMP instead of MLS or RMDS | ResNet-18 · CIFAR-100 Near-OOD and Far-OOD | AUROC: 81.05 -> 79.40; 82.92 -> 79.41 | Hurt |
| Use IF-COMP instead of ACNML-EKFAC | pNML output distribution inference · MNIST LeNet, CIFAR-10 ResNet18, and CIFAR-100 ResNet18 | inference time: 7.34 $\times$; 14.71 $\times$; 12.4 $\times$ | Helped |
| Use temperature-scaled BIF rather than the standard temperature β=1 (ablation) | IF-COMP · CIFAR-10 human-noise mislabel detection | AUROC | Helped |
| Tune α to prioritize calibration for corruption severity (ablation) | IF-COMP · CIFAR-10C uncertainty calibration | ECE | Mixed |
| Increase α beyond 0.5 (ablation) | IF-COMP · CIFAR-10C uncertainty calibration | accuracy | Hurt |
| Vary β while holding α at 0.15 (ablation) | IF-COMP · CIFAR-10C uncertainty calibration | ECE | Helped |
Surprising findings
- Hindsight-optimal retraining from scratch memorizes all added labels, producing a uniform pNML distribution that is useless, so the ground-truth retraining cannot serve as a reference.
- On CIFAR-100, IF-COMP fails to detect human and data-dependent label noise, performing at chance like other baselines.
Limitations
- Requires tuning the temperature beta per task and dataset (e.g., beta=0.001 for CIFAR-10 OOD and mislabel detection), with validation on held-out OOD data.
- Complexity estimates degrade on far-OOD MNIST for ground-truth validation and fail on CIFAR-100 human and data-dependent noise.
- Depends on EKFAC approximation of the Fisher and a first-order linearization, with a log(1+x) approximation valid only for large n.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 MNIST CIFAR-10C OpenOOD benchmark datasets
Benchmarks: CIFAR-10C CIFAR-10 CIFAR-100 MNIST OpenOOD
Metrics: expected calibration error AUROC Pearson correlation inference time test accuracy
Baselines: ACNML Deep Ensembles SWA SWAG-D MC-Dropout Trac-IN EL2N GraNd Self-IF OpenOOD methods (MSP, ODIN, Energy, MLS, KNN, GradNorm, RMDS, VIM, Gram)
Builds on: pNML distribution proximal Bregman objective influence functions ACNML TRAK
Closest papers by meaning
- BoltzNCE: Learning likelihoods for Boltzmann Generation with Stochastic Interpolants and Noise Contrastive Estimation (NeurIPS 2025 · similarity 0.84)
- Uncertainty Quantification with the Empirical Neural Tangent Kernel (NeurIPS 2025 · similarity 0.84)
- Provable Uncertainty Decomposition via Higher-Order Calibration (ICLR 2025 · similarity 0.84)
- Parameter Expanded Stochastic Gradient Markov Chain Monte Carlo (ICLR 2025 · similarity 0.84)
- Neural Conditional Probability for Uncertainty Quantification (NeurIPS 2024 · similarity 0.84)
- Data Mixture Optimization: A Multi-fidelity Multi-scale Bayesian Framework (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.