Research map / Probabilistic and scientific modeling / Molecules and proteins

Unified Analysis of Continuous Weak Features Learning with Applications to Learning from Missing Data

ICML 2025 · Probabilistic and scientific modeling › Molecules and proteins · approach: Sparse concepts and hypothesis testing

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Gives a risk-based formulation, error bounds, and sequential/iterative consistency conditions for continuous weak features learning (WFL) with MSE-based feature estimation, extending discrete WFL and applying it to ItR with missing continuous features.

Problem

Theoretical analysis of learning with continuous weak features (missing, noisy, or interval-observed continuous inputs): how feature estimation model g and label predictor f interact, and when sequential or iterative learning is consistent, which discrete WFL theory cannot handle.

Approach

Replace the 0-1 estimation loss of discrete WFL with MSE, model g as a Gaussian-noise probabilistic model N(g(xo), σ²I) and bound the risk gap via Hellinger distance, yielding a continuous WFL objective and learning-algorithm class (LAC-cWFL) with finite-sample bounds and consistency results for sequential and iterative learning.

Key results

• Experiments on four OpenML datasets show that the generalization error R_{l,g}(f) decreases faster with more training samples when the MSE of g is lower (MSE from 0.001 to 0.5).
• The decrease in the observed generalization error follows a similar trend to the Theorem 4.3 error bound across MSE settings of g (0.005 to 0.5).
• Theory shows sequential learning is consistent under conditions (finite-precision WF values, consistent g learning, vanishing Rademacher complexity), and iterative learning is consistent under additional Lipschitz and Rademacher conditions.

Measured effects reported in the paper

Surprising findings

• The error bound does not account for feature importance of WFs, so discrepancies with observed generalization error remain.

Limitations

• The error bound ignores the importance of WFs in predicting Y, causing discrepancies with observed generalization error.
• Theory assumes Rademacher complexity of order O(1/sqrt(n)) and sufficiently expressive hypothesis classes with realizability.
• Experiments cover only MCAR missingness with a 50% missing rate and synthetic control of g's MSE.

Experimental setup

Datasets: hls4ml_lhc_jets_hlf electricity mv run-or-walk information

Benchmarks: Jets Electricity Mv Run-or-Walk

Metrics: expected logistic risk R_{l,g}(f) error bound value

Builds on: weak features learning (discrete WFL) complementary features learning impute-then-regress structural risk minimization

Closest papers by meaning

• A Unified Framework for Generalization Error Analysis of Learning with Arbitrary Discrete Weak Features (ICML 2025 · similarity 0.93)
• Random features models: a way to study the success of naive imputation (ICML 2024 · similarity 0.83)
• A General Framework for Learning from Weak Supervision (ICML 2024 · similarity 0.83)
• Feature Importance Metrics in the Presence of Missing Data (ICML 2025 · similarity 0.83)
• Weak Supervision Performance Evaluation via Partial Identification (NeurIPS 2024 · similarity 0.83)
• Prediction models that learn to avoid missing values (ICML 2025 · similarity 0.82)