Research map / Label-efficient and robust learning / Recognition and long-tail

Rethinking Learning from Label Proportions via Moment Matching

NeurIPS 2026 · Label-efficient and robust learning › Recognition and long-tail · approach: Partial and multi-label learning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LLP-MM, which casts learning from label proportions as matching classifier-induced multi-order factorial moments to those computed from observed bag proportions, under a structured-bag assumption linked to the Poisson multinomial distribution.

Problem

Learning from label proportions (LLP) typically assumes randomly assembled bags, which is violated in real structured bag settings; existing methods perform suboptimally under non-random bag construction and proportion matching ignores higher-order statistics.

Approach

Under i.i.d. bags with conditionally independent instance labels, bag label counts follow a Poisson multinomial distribution; the method matches multi-order normalized factorial moment tensors of classifier predictions to those derived from observed label proportions, generalizing proportion matching (first order) and tightening the minimizer set with higher orders.

Key results

• LLP-MM ranks first in average accuracy across all nine dataset-bag-construction settings, e.g., 84.21% average on CIFAR-10 random bag vs 82.68% for LLP-PVC (second).
• On CIFAR-10 cluster bags, LLP-MM reaches 87.67% average accuracy vs 85.96% for PM.
• On KU-Optofil PBC, LLP-MM (s=8) reaches 69.75% Macro-F1, exceeding LLP-PVC by 11.33 points.

Surprising findings

• Several recent methods (EasyLLP, GeneralUPM) collapse to near-chance accuracy under cluster bags due to negative empirical losses.
• LLP-FC reaches 72.65% on CIFAR-10 cluster bags of size 128 but drops to about 50% at size 32, showing non-monotone behavior.

Limitations

• Complexity still grows combinatorially with maximum moment order s, so higher orders are expensive for many classes.
• Consistency to the Bayes classifier is not established for structured bags; the generalization result does not yield classifier consistency.
• Lacks standardized real-world multiclass LLP benchmarks; the semi-real evaluation uses a single public dataset.

Experimental setup

Datasets: CIFAR-10 CIFAR-100 miniImageNet KU-Optofil PBC

Benchmarks: CIFAR-10 CIFAR-100 miniImageNet KU-Optofil PBC

Metrics: accuracy macro-F1 balanced accuracy weighted-F1 runtime per epoch

Baselines: PM DSQ LLP-PVC LLP-FC ROT EasyLLP GeneralUPM FlowLLP

Builds on: proportion matching EasyLLP LLP-PVC Poisson multinomial distribution

Closest papers by meaning

• Learning from Label Proportions: Bootstrapping Supervised Learners via Belief Propagation (ICLR 2024 · similarity 0.91)
• Nearly Optimal Sample Complexity for Learning with Label Proportions (ICML 2025 · similarity 0.90)
• Forming Auxiliary High-confident Instance-level Loss to Promote Learning from Label Proportions (CVPR 2025 · similarity 0.90)
• Learning from Noisy Labels via Conditional Distributionally Robust Optimization (NeurIPS 2024 · similarity 0.86)
• Learning with Partial-Label and Unlabeled Data: A Uniform Treatment for Supervision Redundancy and Insufficiency (ICML 2024 · similarity 0.86)
• Addressing Multi-Label Learning with Partial Labels: From Sample Selection to Label Selection (AAAI 2025 · similarity 0.86)