Research map / Large language models / Agents, code and math
Time Travel is Cheating: Going Live with DeepFund for Real-Time Fund Investment Benchmarking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces DeepFund, a live, leakage-free multi-agent benchmark that evaluates nine LLMs on real-time fund investment (Berkshire top-5 holdings, Mar 17–Apr 17 2025), finding most lose money and only Grok 3 mini is profitable.
Problem
Existing LLM trading benchmarks rely on historical back-testing, which lets models "time travel" by exploiting future information in pretraining data, inflating performance estimates.
Approach
Evaluate LLM-driven fund managers forward in time on live market data published after each model's knowledge cutoff, using a multi-agent workflow (financial planner, six analyst types, portfolio manager) driven by a single LLM backend.
Key results
- Only Grok 3 mini Beta achieved positive cumulative return (+1.1%) over Mar 17–Apr 17 2025; DeepSeek-V3 (-5.7%) and Claude 3.7 Sonnet (-3.7%) incurred losses.
- Validity rates: 4144 of 4320 signals (96%) and 1059 of 1080 trading decisions (98%) were produced successfully.
- Evaluation cost about $100 in total.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Evaluate LLM fund strategies in live market conditions rather than relying on retrospective back-testing | LLM fund investment strategies · DeepFund live trading, March 17–April 17, 2025 | CR (%): +1.1; -1.9; -3.7; -4.3; -5.7; -5.9; -6.7; -7.5; -8.1 | Mixed |
| Maintain a higher cash reserve and dynamically rebalance positions | Grok 3 mini Beta fund strategy · DeepFund live trading during tariff-driven market volatility | CR (%): +1.1 | Helped |
| Invest nearly 90% of initial cash and maintain low cash reserves | DeepSeek-V3 fund strategy · DeepFund live trading during tariff-driven market volatility | CR (%): -5.7 | Hurt |
| Use a passive Buy & Hold strategy instead of active LLM trading | LLM fund strategies · DeepFund live trading, March 17–April 17, 2025 | CR (%): -3.09; -1.58; -2.94; -3.62; -5.6; -4.41; -4.86; -3.90; -5.37 | Mixed |
Surprising findings
- Most LLMs underperformed a passive Buy & Hold strategy, which was more resilient to market fluctuations.
- DeepSeek's low cash reserve and concentrated, momentum-driven trading hampered loss mitigation during the tariff downturn.
Limitations
- Only US stock market and a single five-stock portfolio; no transaction fees or execution frictions modeled.
- Short evaluation window (24 trading days) during a volatile market may skew results.
Experimental setup
Datasets: Yahoo Finance Alpha Vantage Berkshire Hathaway top-5 holdings price, news, insider, policy and macro data (live)
Benchmarks: DeepFund live trading environment
Metrics: Cumulative Return Cumulative Return at Buy & Hold Sharpe Ratio Maximum Drawdown Win Rate Beta Alpha
Baselines: Buy & Hold strategy S&P 500
Builds on: InvestorBench FinRL-Meta LangChain ChatBot Arena
Closest papers by meaning
- FinBen: A Holistic Financial Benchmark for Large Language Models (NeurIPS 2024 · similarity 0.87)
- FinSearchComp: Towards a Realistic, Expert-Level Evaluation of Financial Search and Reasoning (ICLR 2026 · similarity 0.85)
- Win Fast or Lose Slow: Balancing Speed and Accuracy in Latency-Sensitive Decisions of LLMs (NeurIPS 2025 · similarity 0.84)
- MarS: a Financial Market Simulation Engine Powered by Generative Foundation Model (ICLR 2025 · similarity 0.84)
- A Theory of Training Profit-Optimal LLMs (NeurIPS 2026 · similarity 0.84)
- FinCon: A Synthesized LLM Multi-Agent System with Conceptual Verbal Reinforcement for Enhanced Financial Decision Making (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.