Research map / Large language models / Agents, code and math
BizFinBench.v2: Towards Reliable LLMs in Finance via Real-User Data and Offline/Online Bilingual Evaluation
Xin Guo, Rongjunchen Zhang, Guilong Lu, Xuntao Guo, Jia Shuai, Zhi Yang, Liwen Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use chain-of-thought prompting instead of zero-shot (ablation) | Claude-Sonnet-4 · BizFinBench.v2 average | accuracy (%): 39.5 -> 13.7 | Hurt |
| use chain-of-thought prompting instead of zero-shot (ablation) | GPT-5 · BizFinBench.v2 average | accuracy (%): 61.5 -> 54.6 | Hurt |
| use chain-of-thought prompting instead of zero-shot (ablation) | Gemini-3 · BizFinBench.v2 average | accuracy (%): 61.3 -> 56.5 | Hurt |
| use chain-of-thought prompting instead of zero-shot (ablation) | DeepSeek-R1 · BizFinBench.v2 average | accuracy (%): 41.5 -> 50.4 | Improved |
| use chain-of-thought prompting instead of zero-shot (ablation) | 21 evaluated LLMs · BizFinBench.v2 | accuracy (%) | Hurt |
| use financial-domain fine-tuned LLM instead of general model of same scale | Dianjin-R1 32B vs Qwen3-32B · BizFinBench.v2 average (zero-shot) | accuracy (%): 41.3 -> 35.7 | Hurt |
| use larger commercial LLM for portfolio allocation vs quantitative strategies | LLM investment simulation · Portfolio Asset Allocation | total return: DeepSeek-R1 +47.34% vs Momentum Top5 101.71% | Mixed |
| compare best LLM with financial experts | GPT-5 · BizFinBench.v2 average | accuracy (%): 61.5 vs 84.8 | Hurt |
Closest papers by meaning
- FinBen: A Holistic Financial Benchmark for Large Language Models (NeurIPS 2024 · similarity 0.92)
- FinSearchComp: Towards a Realistic, Expert-Level Evaluation of Financial Search and Reasoning (ICLR 2026 · similarity 0.88)
- Time Travel is Cheating: Going Live with DeepFund for Real-Time Fund Investment Benchmarking (NeurIPS 2025 · similarity 0.88)
- EnterpriseBench: Evaluating LLM Agents on End-to-End Spreadsheet Tasks in Finance (NeurIPS 2026 · similarity 0.86)
- FinRpt: Dataset, Evaluation System and LLM-based Multi-agent Framework for Equity Research Report Generation (AAAI 2026 · similarity 0.85)
- FinMMDocR: Benchmarking Financial Multimodal Reasoning with Scenario Awareness, Document Understanding, and Multi-Step Computation (AAAI 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
