Research map / Large language models / Instruction and fine-tuning
PIXIU: A Comprehensive Benchmark, Instruction Dataset and Large Language Model for Finance
Qianqian Xie, Weiguang Han, Xiao Zhang, Yanzhao Lai, Min Peng, Alejandro Lopez-Lira, Jimin Huang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| instruction-tune LLaMA-7B on financial NLP data (FinMA-7B) vs Vicuna-7B | LLaMA-7B · FPB | F1: 0.29 -> 0.94 | Improved |
| scale FinMA from 7B to 30B (ablation) | FinMA · ConvFinQA | EM accuracy: 0.25 -> 0.40 | Improved |
| scale FinMA from 7B to 30B (ablation) | FinMA · FinQA | EM accuracy: 0.06 -> 0.11 | Improved |
| scale FinMA from 7B to 30B (ablation) | FinMA · FPB | F1: 0.94 -> 0.88 | Hurt |
| scale FinMA from 7B to 30B (ablation) | FinMA · BigData22 | Accuracy: 0.45 -> 0.47 | No clear effect |
| add prediction task data to NLP instruction data (FinMA-7B-full vs FinMA-7B) (ablation) | FinMA-7B · ACL18 | Accuracy: 0.49 -> 0.51 | Improved |
| add prediction task data to NLP instruction data (FinMA-7B-full vs FinMA-7B) (ablation) | FinMA-7B · CIKM18 | Accuracy: 0.43 -> 0.50 | Improved |
| add prediction task data to NLP instruction data (FinMA-7B-full vs FinMA-7B) (ablation) | FinMA-7B · FPB | F1: 0.94 -> 0.94 | No clear effect |
| fine-tune on prediction tasks only (FinMA-7B-trade) vs NLP-only FinMA-7B (ablation) | FinMA-7B · FPB | F1: 0.94 -> 0.03 | Hurt |
| fine-tune on prediction tasks only (FinMA-7B-trade) vs NLP-only FinMA-7B (ablation) | FinMA-7B · BigData22 | Accuracy: 0.45 -> 0.45 | No clear effect |
| classification-only fine-tuning (FinMA vs Vicuna-7B) | LLaMA-7B · EDTSum | Rouge-1: 0.22 -> 0.09 | Hurt |
Closest papers by meaning
- FinBen: A Holistic Financial Benchmark for Large Language Models (NeurIPS 2024 · similarity 0.92)
- FinSearchComp: Towards a Realistic, Expert-Level Evaluation of Financial Search and Reasoning (ICLR 2026 · similarity 0.88)
- FinMMDocR: Benchmarking Financial Multimodal Reasoning with Scenario Awareness, Document Understanding, and Multi-Step Computation (AAAI 2026 · similarity 0.88)
- A Theory of Training Profit-Optimal LLMs (NeurIPS 2026 · similarity 0.87)
- Time Travel is Cheating: Going Live with DeepFund for Real-Time Fund Investment Benchmarking (NeurIPS 2025 · similarity 0.86)
- STEER-ME: Assessing the Microeconomic Reasoning of Large Language Models (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
