Research map / Reinforcement learning / Visual and representation RL
Logic-Q: Improving Deep Reinforcement Learning-based Quantitative Trading via Program Sketch-based Tuning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Logic-Q, which tunes frozen DRL trading policies post-hoc with a lightweight, Bayesian-optimized program sketch that identifies market trends and adjusts softmax temperature or ensemble weights.
Problem
DRL quantitative trading strategies fail to identify market trends, causing missed opportunities and large drawdowns during market crashes; abstract expert knowledge of trends is hard to quantify.
Approach
Embed abstract market-trend knowledge as a program sketch with numeric holes (volatility, downside risk, growth rate conditions) and use it to post-hoc tune a frozen DRL policy, either by softmax temperature scaling or by ensemble weighting, optimized with Bayesian optimization on validation data.
Key results
- On US stock cash trading, Logic-Q achieves 18.1% AR and Sharpe ratio 1.097, versus 14.2% AR and 0.816 for the best prior baseline AlphaMix.
- On order execution, Logic-Q reaches Price Advantage 4.33 bps and ARR 11.53%, improving over the OPD baseline's 3.41 bps and 8.97%.
- On US margin trading, Logic-Q's maximum drawdown is -26.1% versus -34.3% for AlphaMix (best baseline MD).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the market trend-aware program sketch to policy tuning (ablation) | Logic-Q single-model policy · Order execution (CSI 800) | PA (‱): 3.50 ± 0.29 -> 4.33 ± 0.25 | Helped |
| augment the OPD input state with market features | OPD · Order execution (CSI 800) | PA (‱): 3.41 ± 0.22 -> 3.07 ± 0.21 | Hurt |
| add the market trend-aware program sketch to policy tuning (ablation) | Logic-Q ensemble · US stock market, cash trading | SR: 0.756 -> 1.097 | Helped |
| add the market trend-aware program sketch to policy tuning (ablation) | Logic-Q ensemble · US stock market, margin trading | SR: 0.716 -> 1.175 | Helped |
| add the market trend-aware program sketch to policy tuning (ablation) | Logic-Q ensemble · HK stock market, cash trading | SR: 0.366 -> 0.624 | Helped |
| add the market trend-aware program sketch to policy tuning (ablation) | Logic-Q ensemble · HK stock market, margin trading | SR: 0.416 -> 1.135 | Helped |
| add the market trend-aware program sketch to policy tuning (ablation) | Logic-Q ensemble · Cryptocurrency market, cash trading | SR: 0.824 -> 1.268 | Helped |
| add the market trend-aware program sketch to policy tuning (ablation) | Logic-Q ensemble · Cryptocurrency market, margin trading | SR: 1.129 -> 1.291 | Helped |
| augment model input with market information | AlphaMix and Sharpe-Ens · Stock trading markets | performance | No clear effect |
| use program sketch-based tuning instead of the baseline strategy | Logic-Q · Market crash case 1, cash trading | MD: -62.6% -> -43.3% | Helped |
| use program sketch-based tuning instead of the baseline strategy | Logic-Q · Market crash case 1, margin trading | MD: -38.7% -> -35.1% | Helped |
Surprising findings
- Simply concatenating market indicators into the DRL input state (OPD (Aug), Sharpe-Ens (Aug), AlphaMix (Aug)) does not improve performance, sometimes degrading it.
Limitations
- Evaluation is limited to two trading tasks and a few markets; the program sketch has a fixed set of five trend categories defined by the authors.
- Program sketch is optimized with Bayesian optimization, not end-to-end, and relies on a small validation set.
Experimental setup
Datasets: CSI 800 minute-level intraday data (order execution) Yahoo Finance end-of-day data (Dow Jones 30, Hang Seng China 50, 10 cryptocurrencies)
Benchmarks: CSI 800 order execution US Dow 30 stock trading Hang Seng China 50 stock trading cryptocurrency trading
Metrics: Price Advantage Annualized Rate of Return Gain-loss Ratio Positive Rate Annualized Return Cumulative Return Annualized Volatility Maximum Drawdown Sharpe ratio
Baselines: TWAP VWAP PPO OPD DDPG A2C Sharpe-Ens AlphaMix Buy and Hold
Builds on: program synthesis by sketching OPD Sharpe-Ens AlphaMix
Closest papers by meaning
- EarnHFT: Efficient Hierarchical Reinforcement Learning for High Frequency Trading (AAAI 2024 · similarity 0.86)
- Solver-Informed RL: Grounding Large Language Models for Authentic Optimization Modeling (NeurIPS 2025 · similarity 0.84)
- Trade in Minutes! Rationality-Driven Agentic System for Quantitative Financial Trading (ICLR 2026 · similarity 0.84)
- AlphaQCM: Alpha Discovery in Finance with Distributional Reinforcement Learning (ICML 2025 · similarity 0.84)
- Adaptive $Q$-Network: On-the-fly Target Selection for Deep Reinforcement Learning (ICLR 2025 · similarity 0.84)
- DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoning (ICLR 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.