Research map / Label-efficient and robust learning / Recognition and long-tail

Deep Probabilistic Supervision for Image Classification

NeurIPS 2026 · Label-efficient and robust learning › Recognition and long-tail · approach: Loss design and classifier theory · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Deep Probabilistic Supervision (DPS), which builds sample-specific Dirichlet-based target distributions by discounted Bayesian updating of the model's own predictions, reducing reliance on hard labels for better accuracy, calibration, and label-noise robustness.

Problem

Hard one-hot targets in supervised classification promote overconfidence and poor calibration, and existing self-distillation methods remain anchored to hard labels without explicitly modeling predictive uncertainty, making them sensitive to label noise.

Approach

Treats the network's per-epoch predictions as evidence for a Dirichlet posterior over class probabilities, updated recursively with a discount factor (recursive Bayesian estimation), so training targets are built only from the model's own predictions after initialization; related methods (EMA, PS-KD, DLB, TE) are shown to be special cases.

Key results

• DPS improves ImageNet accuracy for DenseNet-264 by about 2.2 pp over baseline (78.79% vs 76.55%) and ResNet-152 by about 1.5 pp.
• On CIFAR-100 with ResNet-50, DPS lowers ECE by roughly 40% relative to the best related method, and reaches 1.33% ECE with tuned hyperparameters (validation set).
• DPS+ (with contrastive loss) reports state-of-the-art accuracy under label noise on CIFAR-10 with PreAct ResNet-18, matching or exceeding OT-Filter, CSOT, and DULC.

Surprising findings

• The hyperparameters minimizing ECE do not coincide with those maximizing accuracy (lowest ECE at gamma=0.97, c=2000; highest accuracy at gamma=0.96, c=1000).
• DPS yields tighter, less overlapping penultimate-layer clusters than conventional training, despite promoting sample-specific information.

Limitations

• Requires selecting discount factor gamma and prior strength c; poor choices (e.g., gamma=0.9, c=50) degrade performance.
• Interaction with different augmentation and regularization schemes is not fully studied; CutOut underperforms on TinyImageNet.
• Label noise results rely on a sharpening parameter tau and a contrastive term (DPS+) with per-noise-level tuned lambda_c.

Experimental setup

Datasets: CIFAR-10 CIFAR-100 Tiny ImageNet ImageNet CIFAR-10-C CIFAR-10-P SVHN

Benchmarks: CIFAR-10 CIFAR-100 TinyImageNet ImageNet CIFAR-10-C CIFAR-10-P SVHN OOD detection

Metrics: test accuracy Expected Calibration Error NLL mean Flip Probability AUROC

Baselines: Baseline (standard training) Temporal Ensembling DLB PS-KD Knowledge Distillation Label Smoothing Symmetric Cross Entropy MixUp OT-Filter CSOT DULC MDCA

Builds on: Temporal Ensembling PS-KD DLB Knowledge Distillation label smoothing Mean Teacher

Closest papers by meaning

• Probabilistic Self-supervised Representation Learning via Scoring Rules Minimization (ICLR 2024 · similarity 0.87)
• Lightweight Probabilistic Deep Networks (CVPR 2018 · similarity 0.87)
• A critical analysis of self-supervision, or what we can learn from a single image (ICLR 2020 · similarity 0.87)
• SURE: SUrvey REcipes for building reliable and robust deep networks (CVPR 2024 · similarity 0.86)
• Bayesian Exploration of Pre-trained Models for Low-shot Image Classification (CVPR 2024 · similarity 0.85)
• Training Deep Neural Networks with Virtual Smoothing Classes (AAAI 2025 · similarity 0.85)