Research map / Label-efficient and robust learning / Recognition and long-tail

Unified Approach for Weakly Supervised Multicalibration

NeurIPS 2026 · Label-efficient and robust learning › Recognition and long-tail · approach: Partial and multi-label learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Derives decontamination-based estimators of multicalibration error under PU, UU and Pconf weak supervision, with finite-sample uniform guarantees, and proposes WLMC, a weak-label boosting post-hoc recalibration method, evaluated on tabular, CelebA and CivilComments data.

Problem

Multicalibration error estimation and post-hoc correction require clean input-label pairs, which are unavailable in weakly supervised learning settings such as PU, UU and Pconf learning.

Approach

Treat each multicalibration witness moment as a signed residual functional and rewrite it via the contamination-matrix (decontamination) principle of WSL, so corrected empirical witness moments can audit and drive post-hoc recalibration from weak labels only.

Key results

• Weak PU/UU multicalibration estimates are mostly aligned with oracle MC on tabular data, while Pconf shows several deviations.
• Weak-data post-processing reduced oracle MC in 683/720 runs for the (weak,weak) setting.
• On CelebA and CivilComments, weak-label post-processing improves ECE and MC without large accuracy drops; Platt scaling is strong for ECE.

Surprising findings

• Clean (PN,PN) post-hoc correction does not always improve MC, partly because PN baselines often already have small violations.
• Pconf weak estimates deviate from oracle MC on real data, attributed to inherited error in confidence values from a separate probabilistic model.

Limitations

• Requires fresh weak samples at each step of WLMC.
• Needs known or pre-estimated class priors and mixture proportions.
• Restricted to binary labels.

Experimental setup

Datasets: ACS Income UCI Credit Default HMDA MEPS CelebA CivilComments

Benchmarks: ACS Income UCI Credit Default HMDA MEPS CelebA CivilComments

Metrics: multicalibration error ECE accuracy

Baselines: Platt scaling temperature scaling PU-ECE

Builds on: unified risk-analysis for weakly supervised learning (contamination matrix) multicalibration (Hebert-Johnson et al.) PU-ECE Platt scaling temperature scaling

Closest papers by meaning

• Bridging Multicalibration and Out-of-distribution Generalization Beyond Covariate Shift (NeurIPS 2024 · similarity 0.87)
• Multi-Resolution Weak Supervision for Sequential Data (NeurIPS 2019 · similarity 0.86)
• Consistent One-vs-All Losses Robust to Misspecification of the Weak Label Transition Model (NeurIPS 2026 · similarity 0.86)
• Weak Supervision Performance Evaluation via Partial Identification (NeurIPS 2024 · similarity 0.84)
• When is Multicalibration Post-Processing Necessary? (NeurIPS 2024 · similarity 0.84)
• LaSCal: Label-Shift Calibration without target labels (NeurIPS 2024 · similarity 0.83)