Research map / Reinforcement learning / Policy gradient and actor-critic
AlphaQCM: Alpha Discovery in Finance with Distributional Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AlphaQCM, a distributional RL method (IQN quantiles + DQN Q-network + quantiled conditional moment variance bonus) that searches for synergistic formulaic alpha pools on Chinese A-share and S&P 500 data.
Problem
Discovering synergistic formulaic alphas for stock return prediction is cast as an MDP that is non-stationary (reward function changes as the alpha pool updates) and reward-sparse, which hampers existing RL approaches such as AlphaGen.
Approach
Learns Q via a DQN and quantiles via IQN, then estimates an unbiased variance from possibly biased quantiles using the quantiled conditional moment (QCM) method; the variance serves as an exploration bonus in action selection (Q + λ√variance).
Key results
- AlphaQCM achieves out-of-sample mean IC of 8.49% on CSI 300, 9.55% on CSI 500, and 9.16% on Market, the highest among all compared methods (AlphaGen: 8.13%, 8.08%, 6.04%).
- On S&P 500, AlphaQCM reaches 8.46% IC versus 7.48% for AlphaGen.
- Ablation with IQN backbone: QCM variance bonus gives 8.49%/9.55%/9.16% vs 7.17%/8.58%/7.04% with no variance and 6.16%/8.75%/8.42% with vanilla quantile variance.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace AlphaGen with AlphaQCM | formulaic alpha discovery · CSI 300 | IC: 8.13% -> 8.49% | Helped |
| replace AlphaGen with AlphaQCM | formulaic alpha discovery · CSI 500 | IC: 8.08% -> 9.55% | Helped |
| replace AlphaGen with AlphaQCM | formulaic alpha discovery · Market | IC: 6.04% -> 9.16% | Helped |
| add QCM variance bonus instead of no variance bonus (ablation) | AlphaQCM with QRDQN backbone · CSI 300 | IC: 6.96% -> 7.59% | Helped |
| add QCM variance bonus instead of no variance bonus (ablation) | AlphaQCM with QRDQN backbone · CSI 500 | IC: 8.54% -> 9.08% | Helped |
| add QCM variance bonus instead of no variance bonus (ablation) | AlphaQCM with QRDQN backbone · Market | IC: 7.06% -> 9.12% | Helped |
| replace vanilla quantile-based variance with QCM variance (ablation) | AlphaQCM with IQN backbone · CSI 300 | IC: 6.16% -> 8.49% | Helped |
| replace vanilla quantile-based variance with QCM variance (ablation) | AlphaQCM with IQN backbone · CSI 500 | IC: 8.75% -> 9.55% | Helped |
| replace vanilla quantile-based variance with QCM variance (ablation) | AlphaQCM with IQN backbone · Market | IC: 8.42% -> 9.16% | Helped |
| use IQN instead of QRDQN as the distributional RL backbone (ablation) | AlphaQCM with QCM variance · CSI 300 | IC: 7.59% -> 8.49% | Helped |
| use expert formulaic alphas to initialize replay memory (ablation) | AlphaQCM at 100% training · CSI 300 | IC: 8.49% -> 8.17% | Hurt |
| use expert formulaic alphas to initialize replay memory (ablation) | AlphaQCM at 100% training · CSI 500 | IC: 9.55% -> 8.96% | Hurt |
Surprising findings
- Domain knowledge initialization (replay memory seeded with Alpha101 trajectories) gives early gains but lower final IC than training from scratch.
- Mutual IC filter improves GP performance on CSI 300 and CSI 500 but hurts it on the full Market dataset.
Limitations
- Results depend on the choice of alpha pool size P and the risk-preference parameter λ, tuned on validation data.
- The paper notes the Q estimate remains biased under non-stationarity; QCM only gives unbiased variance, not Q.
- Evaluated only on Chinese A-share and S&P 500 with 20-day forward returns; the claimed generality to other non-stationary or reward-sparse environments is not tested.
Experimental setup
Datasets: CSI 300 CSI 500 Market (all Shanghai/Shenzhen A-shares) S&P 500 Alpha101 formulas
Benchmarks: CSI 300 CSI 500 Market (Chinese A-shares) S&P 500
Metrics: Information Coefficient (IC) IC standard deviation across 10 seeds
Baselines: Alpha101 MLP XGBoost LightGBM GP w/o filter GP w/ filter PPO w/ filter AlphaGen QRDQN
Builds on: IQN DQN QCM AlphaGen QRDQN
Closest papers by meaning
- AlphaPareto: Formulaic Alpha Discovery with LLM-Guided Multi-Objective Reinforcement Learning (NeurIPS 2026 · similarity 0.92)
- AlphaForge: A Framework to Mine and Dynamically Combine Formulaic Alpha Factors (AAAI 2025 · similarity 0.88)
- R&D-Agent-Quant: A Multi-Agent Framework for Data-Centric Factors and Model Joint Optimization (NeurIPS 2025 · similarity 0.87)
- AlphaPROBE: Alpha Mining via principled retriveval and on-graph biased evolution (NeurIPS 2026 · similarity 0.86)
- Robust Reinforcement Learning in Finance: Modeling Market Impact with Elliptic Uncertainty Sets (NeurIPS 2025 · similarity 0.85)
- Augmented Lagrangian Risk-constrained Reinforcement Learning for Portfolio Optimization (Student Abstract) (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.