Research map / Learning theory and algorithms / Prediction and decision losses
Adaptive Passive-Aggressive Framework for Online Regression with Side Information
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes APAS, an adaptive passive-aggressive online regression framework that uses a proximal side-information term and adapts the PA threshold epsilon via online gradient descent, with an SCA-based fast solver and O(sqrt(T)) regret, evaluated on synthetic and S&P 500/NASDAQ 100 enhanced index tracking.
Problem
Choosing the PA threshold in online linear regression is hard, and standard PA ignores side information (e.g., return or switching cost) beyond tracking accuracy.
Approach
Integrates a convex side-information function h_t(w) into the PA update via a proximal step, and adaptively updates the threshold epsilon by online (sub)gradient descent on the Moreau envelope loss, with an SCA solver that avoids an interior-point solve.
Key results
- APAS attains higher excess cumulative return than PA and SLAIT at matched tracking error on synthetic data, with tracking error tuned via lambda.
- Theorem 2 gives an O(sqrt(T)) regret bound for the adaptive threshold scheme under bounded-subderivative assumptions, without convexity of the loss.
- The SCA-based solver (Algorithm 2) is reported to be about 100x faster than CVXR and ADMM and 10x faster than PGD for high-dimensional data (N=1000).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use APAS with side information and an adaptive threshold instead of PA | Online index tracking · Synthetic, S&P 500, and NASDAQ 100 | Excess cumulative return at comparable tracking error | Helped |
| Use APAS with a larger trade-off parameter λ | Online index tracking · S&P 500 and NASDAQ 100 | Excess cumulative return | Helped |
| Use APAS with a larger trade-off parameter λ | Online index tracking · S&P 500 and NASDAQ 100 | Tracking error | Hurt |
| Replace a fixed ε in PAS with adaptive ε updates (ablation) | PAS on synthetic data · Synthetic | Tracking error and excess cumulative return | Helped |
| Use Student’s t-distributed data instead of Gaussian data (ablation) | APAS · Synthetic, tracking error | Tracking error | No clear effect |
| Use Student’s t-distributed noise instead of Gaussian noise (ablation) | APAS · Synthetic, excess cumulative return | Excess cumulative return | Hurt |
| Use Student’s t-distributed data instead of Gaussian data (ablation) | APAS · Synthetic, excess cumulative return | Excess cumulative return | Helped |
| Use group Lasso rather than other listed side-information functions (ablation) | APAS efficient algorithm · Algorithm 2, larger dimensions N | CPU time | Hurt |
Surprising findings
- Heavy-tailed data yields a modest improvement in excess cumulative return for APAS, attributed to more positive outliers in side performance.
Limitations
- Regret bound relies on bounded subderivative and domain assumptions (Assumptions 1-2) and only holds for the threshold adaptation, not jointly for the weights.
- Experiments are limited to enhanced index tracking on linear regression with stock returns and a small number of datasets.
Experimental setup
Datasets: S&P 500 stock data from Yahoo! Finance NASDAQ 100 stock data from Yahoo! Finance synthetic Gaussian and Student-t data
Benchmarks: S&P 500 index tracking NASDAQ 100 index tracking synthetic index tracking
Metrics: magnitude of daily tracking error (MDTE) excess cumulative return CPU time convergence gap
Baselines: PA SLAIT-ETE SLAIT-DR PAS with fixed epsilon CVXR PGD ADMM
Builds on: Passive-Aggressive (Crammer et al., 2006) SCA (Scutari et al., 2013) SLAIT (Benidis et al., 2017)
Closest papers by meaning
- Stabilizing Linear Passive-Aggressive Online Learning with Weighted Reservoir Sampling (NeurIPS 2024 · similarity 0.88)
- AKORN: Adaptive Knots generated Online for RegressioN splines (ICML 2025 · similarity 0.82)
- Online Linear Regression in Dynamic Environments via Discounting (ICML 2024 · similarity 0.82)
- Adaptive Data Optimization: Dynamic Sample Selection with Scaling Laws (ICLR 2025 · similarity 0.82)
- Minimax Adaptive Online Nonparametric Regression over Besov spaces (NeurIPS 2025 · similarity 0.82)
- Variance Reduction for Expectations with Diffusion Teachers (NeurIPS 2026 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.