Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Rate-In: Information-Driven Adaptive Dropout Rates for Improved Inference-Time Uncertainty Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Rate-In, an unsupervised inference-time method that adapts MC Dropout rates per layer and per input by controlling mutual-information loss induced by dropout, evaluated on synthetic regression and MedMNIST classification and prostate/liver segmentation.
Problem
Monte Carlo Dropout uses static dropout rates across all layers and inputs at inference, yielding suboptimal uncertainty estimates; prior dropout-rate optimization needs labeled training data and yields fixed inference-time rates.
Approach
Treat dropout as controlled noise injection in each layer's channel and iteratively adjust each layer's dropout rate at inference so that the mutual information loss between layer input and output stays near a threshold, requiring no labels.
Key results
- On TissueMNIST at dropout 0.20, Rate-In achieves accuracy 0.48 (AUARC 0.59) versus 0.09 (0.25) for constant dropout.
- On PathMNIST at dropout 0.20, Rate-In achieves 0.75 (AUARC 0.91) versus 0.41 (0.56) for constant dropout.
- On prostate MRI segmentation (peripheral zone), Rate-In yields DSC change +2.79% at p=0.05 and lower ECE (4.26 vs 5.55 for benchmark at p=0.05 in x10^-3).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Rate-In instead of constant dropout at p=0.05 | ResNet-18 · TissueMNIST | accuracy: .53 -> .68 | Helped |
| Use Rate-In instead of constant dropout at p=0.10 | ResNet-18 · TissueMNIST | AUARC: .37 -> .80 | Helped |
| Use Rate-In instead of constant dropout at p=0.20 | ResNet-18 · PathMNIST | accuracy: .41 -> .75 | Helped |
| Use Rate-In instead of constant dropout at p=0.10 | ResNet-18 · PathMNIST | AUARC: .91 -> .97 | Helped |
| Use Rate-In instead of constant dropout at p=0.20 | ResNet-18 · BloodMNIST | accuracy: .58 -> .82 | Helped |
| Use Rate-In instead of constant dropout at p=0.05 | ResNet-18 · BloodMNIST | AUARC: .99 -> .99 | No clear effect |
| Use Rate-In instead of the benchmark at p=0.05 | nnU-Net · Prostate MRI, peripheral zone | ECE ×10−3: 5.55 -> 4.26 | Helped |
| Use Rate-In instead of the benchmark at p=0.05 | nnU-Net · Prostate MRI, transitional zone | BUC: 0.62 -> 0.84 | Helped |
| Use Rate-In instead of the benchmark at p=0.05 | nnU-Net · Liver CT | DSC (Full Model, % change): 0.00% -> +1.05% | Helped |
| Use Rate-In instead of the benchmark at p=0.10 | nnU-Net · Tumor CT | BUC: 0.53 -> 0.48 | Hurt |
| Increase dropout rate from 0.05 to 0.20 (ablation) | Rate-In · Tumor CT | DSC (Full Model, % change): +1.21% -> -2.59% | Hurt |
| Increase dropout rate from 0.01 to 0.10 (ablation) | Rate-In · Liver CT | ECE ×10−3: 6.73 -> 4.65 | Helped |
Surprising findings
- Fixed-rate MC dropout accuracy degrades sharply at 0.10-0.20 dropout on TissueMNIST and PathMNIST, while Rate-In stays near full-model accuracy.
Limitations
- Mutual information estimation is non-trivial in high-dimensional spaces and relies on approximate estimators
- Added inference-time computation from iterative per-layer rate adjustment
- Hyperparameters (information loss threshold, delta, Nmax) require selection; set from preliminary experiments
Experimental setup
Datasets: synthetic sin regression PathMNIST BloodMNIST TissueMNIST Prostate MRI segmentation Liver Tumor CT segmentation MedMNIST-C
Benchmarks: PathMNIST BloodMNIST TissueMNIST MedMNIST-C Prostate MRI segmentation Liver Tumor CT segmentation synthetic sin regression
Metrics: accuracy AUARC ECE BUC DSC MSE interval width to coverage ratio
Baselines: Fixed Dropout (MC Dropout) Scheduled Dropout Activation-Based Dropout Masksembles MC Batch Normalization
Builds on: MC Dropout adaptive binning mutual information estimator nnU-Net
Closest papers by meaning
- Contextual Dropout: An Efficient Sample-Dependent Dropout Module (ICLR 2021 · similarity 0.86)
- Uncertainty Estimation for Pretrained Medical Image Registration Models via Transformation Equivariance (NeurIPS 2026 · similarity 0.84)
- Entropy Minimization without Model Collapse: Mitigating Prediction Bias in Medical Imaging (NeurIPS 2026 · similarity 0.84)
- Hierarchical Uncertainty Estimation for Learning-based Registration in Neuroimaging (ICLR 2025 · similarity 0.84)
- Non-Asymptotic Uncertainty Quantification in High-Dimensional Learning (NeurIPS 2024 · similarity 0.83)
- Transitional Uncertainty with Layered Intermediate Predictions (ICML 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.