Research map / Learning theory and algorithms / Prediction and decision losses
Hypothesis Testing for Class-Conditional Noise Using Local Maximum Likelihood
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes nonparametric hypothesis tests for class-conditional vs uniform label noise using local maximum likelihood logistic regression with anchor points, comparing against the parametric MLE-based test on synthetic XOR data and a smart-home localisation dataset.
Problem
Assessing whether a binary labelled dataset has class-conditional rather than uniform label noise before learning, when the parametric logistic regression assumptions behind existing tests may not hold.
Approach
Replace the parametric logistic regression underlying the anchor-point hypothesis test with local (kernel-weighted) maximum likelihood logistic regression, deriving the asymptotic variance of the posterior estimate at strict and relaxed anchor points to build p-values for the test of alpha = beta.
Key results
- On symmetric XOR, nonparametric test p-values decrease as sample size N and number of anchors k increase, and with stricter anchors.
- On asymmetric XOR, the parametric test's green and purple boxes both go down with N and k, showing the parametric test fails when its linear assumptions are violated, while the nonparametric test still detects noise.
- On the smart-home dataset, including stair RSSI readings in training makes p-values fall below the cutoff as anchors increase (spurious noise detection), while excluding stairs keeps p-values above the cutoff.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase training sample size | Nonparametric test · Symmetric XOR | p-value | Helped |
| increase the number of anchor points | Nonparametric test · Symmetric XOR | p-value | Helped |
| use stricter rather than more relaxed anchor points | Nonparametric test · Symmetric XOR | p-value | Helped |
| increase the difference between noise rates | Nonparametric test · Symmetric XOR | p-value | Helped |
| apply the parametric test when model assumptions do not hold | Parametric test · Asymmetric XOR | p-value | Hurt |
| include stairs readings in training and use them as anchors | Nonparametric test · Smart home dataset | p-value | Hurt |
Surprising findings
- The parametric test also works on symmetric XOR despite its assumptions not holding, attributed to symmetry and overlap of the Gaussians.
- Including stair readings in training yields spurious evidence of class-conditional noise, showing that grouping ambiguous instances into classes can bias the test.
Limitations
- Empty-neighbourhood problem in local likelihood estimation, especially with low data or instances far from data.
- Smoothing bias is not corrected; the authors follow the no-correction option for illustration.
- Requires user-provided anchor points with known or approximately known posterior of 1/2.
Experimental setup
Datasets: synthetic symmetric XOR Gaussian mixture synthetic asymmetric XOR Gaussian mixture smart home RSSI walkaround dataset
Benchmarks: synthetic symmetric XOR synthetic asymmetric XOR smart home dataset
Metrics: p-values under null of uniform label noise
Baselines: parametric hypothesis test for class-conditional label noise
Builds on: parametric logistic regression label noise test (anchor-point based) local likelihood models
Closest papers by meaning
- Enhanced Sample Selection with Confidence Tracking: Identifying Correctly Labeled Yet Hard-to-Learn Samples in Noisy Data (AAAI 2025 · similarity 0.86)
- SimPro: A Simple Probabilistic Framework Towards Realistic Long-Tailed Semi-Supervised Learning (ICML 2024 · similarity 0.86)
- Learning from Label Proportions: Bootstrapping Supervised Learners via Belief Propagation (ICLR 2024 · similarity 0.86)
- Learning from Noisy Labels via Conditional Distributionally Robust Optimization (NeurIPS 2024 · similarity 0.86)
- Estimating Noisy Class Posterior with Part-level Labels for Noisy Label Learning (CVPR 2024 · similarity 0.85)
- CGMatch: A Different Perspective of Semi-supervised Learning (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.