Research map / Label-efficient and robust learning / Noisy labels

Leveraging Data Symmetries to Select an Optimal Subset of Training Data under Label Noise

NeurIPS 2026 · Label-efficient and robust learning › Noisy labels · approach: Noise-aware loss and pairwise methods · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that cutstats subset selection under label noise is limited by k-NN accuracy, which degrades in high dimensions, and that invariant representations (orthogonal/permutation groups, learned via group-invariant or contrastive methods) restore near-Bayes subset selection.

Problem

Cutstats, a k-NN-based classifier-agnostic method for selecting clean subsets of label-noisy data, has unexplored performance in high-dimensional data, where k-NN may not approximate the Bayes classifier.

Approach

Use invariant representations (norm for SO(d), sorting for permutations) inside cutstats's neighbor search and weights so that k-NN accuracy, and hence subset quality, is preserved in high dimensions; theoretically link subset error rates to k-NN class-wise accuracies.

Key results

• On synthetic orthogonal group data (p=0.45 noise), the proposed method reaches 74.31% subset accuracy versus 60.51% for vanilla cutstats.
• On Rotated MNIST at noise 0.6, group-invariant representation gives 89.37% classifier accuracy versus 83.78% for vanilla cutstats.
• On Tetris at noise 0.4, group-invariant representation gives 75.72% classifier accuracy and 100% subset accuracy versus 57.05% and 60.69% for vanilla cutstats.

Surprising findings

• On Rotated MNIST and Tetris at noise 0.4, Noisy-Complete attains 88.18% and 86.63% classifier accuracy, comparable to or above some subset methods, suggesting subset selection does not always help at low noise.

Limitations

• Theoretical guarantees rely on assumptions from prior work (Tsybakov noise, Hölder continuity, conditional independence) and are asymptotic in nature.
• Effectiveness depends on how accurately group invariance is known or learned; invariance error degrades subset accuracy.
• Experiments are limited to a few groups (SO(d), permutations, SE(3)) and small benchmarks.

Experimental setup

Datasets: synthetic orthogonal-invariant data synthetic permutation-invariant data Rotated MNIST Tetris

Benchmarks: Rotated MNIST Tetris synthetic orthogonal group data synthetic permutation group data

Metrics: classifier accuracy subset accuracy

Baselines: Noisy-Complete Random Vanilla Cutstats Entropy Forget Herding

Builds on: cutstats k-NN Bayes consistency under Tsybakov noise group-invariant representation learning contrastive learning

Closest papers by meaning

• Towards a statistical theory of data selection under weak supervision (ICLR 2024 · similarity 0.86)
• What Does It Take to Build a Performant Selective Classifier? (NeurIPS 2025 · similarity 0.85)
• Information-theoretic Limits of Online Classification with Noisy Labels (NeurIPS 2024 · similarity 0.85)
• Structural-Entropy-Based Sample Selection for Efficient and Effective Learning (ICLR 2025 · similarity 0.85)
• Enhanced Sample Selection with Confidence Tracking: Identifying Correctly Labeled Yet Hard-to-Learn Samples in Noisy Data (AAAI 2025 · similarity 0.85)
• Estimation of the Label-Noise Transition Matrix with Performance Guarantees via Selective Classification (NeurIPS 2026 · similarity 0.85)