Research map / Label-efficient and robust learning / Recognition and long-tail
Self-Calibrating Vicinal Risk Minimisation for Model Calibration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Self-Calibrating Vicinal Risk Minimisation (SCVRM), a Gaussian vicinal data augmentation whose soft labels decay with distance from labeled images, to calibrate dense binary classifiers such as salient object detection models.
Problem
Dense binary classification models (e.g., salient object detection) are over-confident and poorly calibrated; existing calibration methods do not explore the image vicinity space to regularise confidence.
Approach
Vicinal augmentation with Gaussian noise of random scale sigma ~ U(0, gamma]; each vicinal image receives a label smoothed toward uniform by a Gaussian function of its distance from the labeled image, so label confidence decreases with distance. Theoretically shown, for logistic regression, to equal VRM plus a regulariser penalising over-confident predictions.
Key results
- On DUTS-TE, SCVRM reduces ECE to 0.78% versus 1.40% for the second-best method ASLP (a 44.3% reduction).
- On 500 out-of-distribution texture images, SCVRM achieves ECE 11.93% and OE 8.26%, compared with 18.31% and 16.37% for ASLP.
- In the ablation, plain VRM does not reduce calibration error over ERM in general, whereas SCVRM lowers DUTS-TE ECE from 3.05% to 0.78%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add distance-based softened labels to vicinal samples (ablation) | ERM/U-Net, ResNet50 · DUTS-TE | ECE: 3.05 -> 0.78 | Helped |
| add distance-based softened labels to vicinal samples (ablation) | ERM/U-Net, ResNet50 · SOD | ECE: 7.07 -> 3.90 | Helped |
| add fixed-standard-deviation Gaussian vicinal augmentation with hard labels (ablation) | ERM/U-Net, ResNet50 · DUTS-TE | ECE: 3.05 -> 3.54 | Hurt |
| add fixed-standard-deviation Gaussian vicinal augmentation with hard labels (ablation) | ERM/U-Net, ResNet50 · SOD | ECE: 7.07 -> 6.60 | Helped |
| use random standard deviation σ ∼ U(0, γ] for vicinal augmentation with hard labels (ablation) | ERM/U-Net, ResNet50 · DUTS-TE | ECE: 3.05 -> 3.18 | Mixed |
| use random standard deviation σ ∼ U(0, γ] for vicinal augmentation with hard labels (ablation) | ERM/U-Net, ResNet50 · SOD | ECE: 7.07 -> 7.35 | Hurt |
| increase exploration radius γ from too-small values; best performance is reported across γ = [1, 10] (ablation) | SCVRM · DUTS-TE | ECE | Mixed |
| vary label-softening scale η; use η in [0.5√d, 2√d] rather than too small or too large values (ablation) | SCVRM · DUTS-TE | ECE | Mixed |
| vary the number M of augmented images sampled per labeled image per epoch across {1, 2, 3, 4, 5} (ablation) | SCVRM · DUTS-TE | ECE | No clear effect |
| apply SCVRM instead of ASLP | SOD calibration methods · DUTS-TE | ECE: 1.40 -> 0.78 | Helped |
| apply SCVRM instead of ASLP | SOD calibration methods · DUT-OMRON | ECE: 1.99 -> 1.64 | Helped |
| apply SCVRM instead of ASLP | SOD calibration methods · PASCAL-S | ECE: 2.31 -> 1.91 | Helped |
Surprising findings
- Plain VRM with a fixed Gaussian kernel does not reduce calibration error over ERM in general and can increase it, while the label-confidence decay is what drives the gains.
Limitations
- Theoretical equivalence to VRM plus regulariser is proven only for a single-layer logistic regression with binary cross-entropy.
- Approximates the L2 distance by sigma*sqrt(d) to avoid computing high-dimensional norms, which is a heuristic approximation.
- Main experiments are limited to salient object detection; other tasks are reported in supplementary material.
Experimental setup
Datasets: DUTS-TR
Benchmarks: DUTS-TE DUT-OMRON PASCAL-S SOD ECSSD HKU-IS Describable Texture Dataset (500 OoD texture images)
Metrics: ECE OE accuracy
Baselines: Brier Loss Temperature Scaling MMCE Label Smoothing Mixup Focal Loss AdaFocal ASLP
Builds on: Vicinal Risk Minimisation Mixup Label Smoothing Expected Calibration Error
Closest papers by meaning
- Reassessing How to Compare and Improve the Calibration of Machine Learning Models (ICLR 2025 · similarity 0.85)
- A Benchmark Study on Calibration (ICLR 2024 · similarity 0.85)
- Divide et Calibra: Multiclass Local Calibration via Vector Quantization (NeurIPS 2026 · similarity 0.85)
- Self-Calibrating Conformal Prediction (NeurIPS 2024 · similarity 0.84)
- Beyond One-Hot Labels: Semantic Mixing for Model Calibration (ICML 2025 · similarity 0.84)
- Uncertainty Weighted Gradients for Model Calibration (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.