Research map / Reinforcement learning / Visual and representation RL

AlphaPareto: Formulaic Alpha Discovery with LLM-Guided Multi-Objective Reinforcement Learning

NeurIPS 2026 · Reinforcement learning › Visual and representation RL · approach: RL with tree search for discovery · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AlphaPareto, an RL framework for formulaic alpha discovery that conditions the policy on a frozen-LLM embedding of the current alpha pool and optimizes a four-objective vector reward (IC, temporal stability, perturbation robustness, diversity) via MAP-based Pareto-regularized scalarization.

Problem

Formulaic alpha discovery via RL is hampered by non-stationary rewards (the reward depends on the evolving alpha pool, which is not in the state) and by optimizing only a single scalar objective (IC), ignoring stability, robustness and diversity of the alpha pool.

Approach

Augment the RL state with a semantic embedding of the current alpha pool obtained from a frozen LLM (Qwen-Embedding-4B) over a structured prompt, and replace the scalar IC reward with a four-dimensional vector reward scalarized online by MAP's palette-based dual weights.

Key results

• AlphaPareto achieves out-of-sample IC of 3.92% on CSI300, 5.70% on CSI800 and 10.10% on Market, versus 3.69%, 5.07%, 9.16% for the second-best method.
• Backtest annualized return of 58.92% with IR 2.62 and Sharpe 2.59, the highest among compared methods; max drawdown -18.12%.
• Random-noise control on CSI800 drops mean IC to 4.22% (std 1.62%), versus 5.70% for AlphaPareto, supporting pool-dependent semantic information.

Surprising findings

• Qwen-Embedding-4B gave the best ICs, and the scaling pattern across LLM sizes (1.5B to 32B) was non-monotonic.
• Adding the LLM component alone reduced IC on CSI300 (2.37% vs 3.69% for AlphaGen), and the combined model gave a slightly lower mean than MORL-only on CSI300 (3.92% vs 3.99%).

Limitations

• Performance may depend on the prompt template, which was not systematically studied.
• Uses a linear mega-alpha combiner, which may exclude useful nonlinear structure.
• Some benchmark comparisons are not statistically significant (e.g., vs AlphaQCM on Market, p=27.6%).

Experimental setup

Datasets: Chinese A-share CSI300 CSI800 Full market S&P 500

Benchmarks: CSI300 CSI800 Market (A-share) S&P 500

Metrics: out-of-sample IC annualized return maximum drawdown information ratio Sharpe ratio temporal stability (RRE) PFS DH paired t-test

Baselines: Alpha158 MLP GP AlphaAgent R&D-Agent-Quant AlphaGen AlphaForge AlphaQCM

Builds on: AlphaGen MaskPPO MAP (Multi-Human-Value Alignment Palette) AlphaQCM AlphaForge

Closest papers by meaning

• AlphaQCM: Alpha Discovery in Finance with Distributional Reinforcement Learning (ICML 2025 · similarity 0.92)
• Pareto-Optimality, Smoothness, and Stochasticity in Learning-Augmented One-Max-Search (ICML 2025 · similarity 0.86)
• AlphaForge: A Framework to Mine and Dynamically Combine Formulaic Alpha Factors (AAAI 2025 · similarity 0.86)
• Discovering Temporally-Aware Reinforcement Learning Algorithms (ICLR 2024 · similarity 0.86)
• AlphaPROBE: Alpha Mining via principled retriveval and on-graph biased evolution (NeurIPS 2026 · similarity 0.85)
• DISCOVER: Automated Curricula for Sparse-Reward Reinforcement Learning (NeurIPS 2025 · similarity 0.85)