Research map / Learning theory and algorithms / Prediction and decision losses
Scale-invariant Optimal Sampling for Rare-events Data and Sparse Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a scale-invariant optimal subsampling probability (P-optimality) for rare-events logistic models with adaptive lasso variable selection, plus a penalized MSCL estimator and a two-step subsampling algorithm.
Problem
Optimal subsampling for rare-events binary data depends on covariate scaling, which can be especially harmful for sparse models with inactive features; variable selection for rare-events data had not been studied.
Approach
Defines the optimal subsampling function by minimizing the asymptotic prediction error (MSPE) of the IPW adaptive lasso, yielding a P-optimal probability that depends only on active features and is invariant to rescaling x; adds a penalized MSCL estimator that is more efficient than the IPW version.
Key results
- P-OS achieves lower eMSE and eMSPE than A-OS and L-OS across three simulated cases at sampling rates 0.004-0.010 with N=500,000.
- On the Font data, P-OS gives significantly higher AUC than A-OS and L-OS, which are similar to each other.
- Optimal sampling takes about 0.77% of the computation time of the full data adaptive lasso.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use P-OS instead of uniform sampling | Adaptive lasso variable selection · Simulation Case C | false negative rate: 0.168(0.017) -> 0.084(0.013) | Helped |
| Use P-OS instead of uniform sampling | Adaptive lasso variable selection · Simulation Case C | false negative rate: 0.100(0.013) -> 0.066(0.011) | Helped |
| Use P-OS instead of uniform sampling | Adaptive lasso variable selection · Simulation Case C | false negative rate: 0.066(0.011) -> 0.046(0.009) | Helped |
| Use P-OS instead of uniform sampling | Adaptive lasso variable selection · Simulation Case C | false negative rate: 0.068(0.011) -> 0.054(0.010) | Helped |
| Use optimal nonuniform sampling instead of uniform sampling | Adaptive lasso subsampling · Simulation | eMSE | Helped |
| Use P-OS instead of A-OS or L-OS | Adaptive lasso subsampling · Simulation | eMSE | Helped |
| Use P-OS instead of A-OS or L-OS | Adaptive lasso subsampling · Simulation | eMSPE | Helped |
| Use nonuniform optimal sampling instead of uniform sampling | Adaptive lasso subsampling · Simulation | eMSPE | Helped |
| Use P-OS instead of uniform sampling | Adaptive lasso · Covtype | AUC | Helped |
| Use P-OS instead of A-OS or L-OS | Adaptive lasso · Font | AUC | Helped |
| Use A-OS instead of uniform sampling | Adaptive lasso · Font | AUC | Hurt |
| Use two-step subsampling instead of full-data adaptive lasso | Adaptive lasso · Simulation Case A | computational time (seconds): 129.62 -> 1.06 | Helped |
Surprising findings
- A-OS can perform no better than uniform sampling when a covariate is rescaled (s=0.01), showing scale-dependence of existing optimal subsampling.
Limitations
- Theory is asymptotic and sparse-model specific; non-asymptotic guarantees and high-dimensional regimes are not covered.
- Optimal probabilities target estimation/prediction accuracy rather than variable selection quality.
- Assumes correctly specified sparse model; no model misspecification analysis.
Experimental setup
Datasets: Covtype (UCI) Character Font Images (UCI) simulated logistic regression data
Benchmarks: Covtype Font
Metrics: empirical median squared error (eMSE) empirical median squared prediction error (eMSPE) AUC false negative rate computational time
Baselines: A-OS L-OS Uniform sampling Full data lasso Full data adaptive lasso
Builds on: optimal subsampling for rare-events data (Wang et al. A/L-optimality) adaptive lasso maximum sampled conditional likelihood Poisson subsampling
Closest papers by meaning
- Optimal Subsampling with Influence Functions (NeurIPS 2018 · similarity 0.88)
- Understanding Randomization in Greedy Model Search (NeurIPS 2026 · similarity 0.86)
- Provable Posterior Sampling with Denoising Oracles via Tilted Transport (NeurIPS 2024 · similarity 0.84)
- Towards a statistical theory of data selection under weak supervision (ICLR 2024 · similarity 0.84)
- Direct Fisher Score Estimation for Likelihood Maximization (NeurIPS 2025 · similarity 0.84)
- Importance Sampling for Nonlinear Models (ICML 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.