Research map / Multi-agent systems and games / LLM agents

FinRpt: Dataset, Evaluation System and LLM-based Multi-agent Framework for Equity Research Report Generation

AAAI 2026 · Multi-agent systems and games › LLM agents · approach: LLM multi-agent collaboration and safety · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Formulates equity research report (ERR) generation, builds the FinRpt Chinese-English dataset (6,825 reports, CSI800) with an 11-metric evaluation system, and proposes FinRpt-Gen, a nine-agent framework trained with SFT and DAPO-based RL.

Problem

Automating equity research report generation from heterogeneous financial data is unexplored, lacking both training data and evaluation metrics that cover financial accuracy, trend prediction, and professionalism.

Approach

Decomposes ERR writing into a multi-agent pipeline (extraction, analysis, prediction agents) with an automated dataset construction pipeline that refines reports against expert reports and enforces rating consistency with ground-truth trends, plus a reward combining rating accuracy and ROUGE for RL on the prediction agent.

Key results

• FinRpt-Gen (Qwen2.5-7B-Instruct-SFT-RL) reaches 55% recommendation accuracy and 49.06 ROUGE-L on 100 test samples, versus 48% for GPT-4o single-model and 50% for Gemini-2.5-Pro.
• Pairwise LLM judging gives the SFT-RL 7B model an average adjusted win rate of 0.69 against baselines, with 0.86 on FN.
• Human evaluation: expert-written ERRs average 4.30 and FinRpt ERRs 4.20 on a 0-5 scale; the GPT-4o judge agreed with human majority in 45/50 pairs (90%).

Surprising findings

• Single-LLM accuracy for buy/sell ratings stays near 33-50% across all models, close to chance for a binary label.
• The SFT-tuned 7B model outperforms GPT-4o and Gemini-2.5-Pro on most automatic metrics, though the reference data were itself generated by GPT-4o within the FinRpt-Gen pipeline.

Limitations

• Dataset reports are generated by GPT-4o within the pipeline and refined by GPT-4o, so the references may be biased toward GPT-4o outputs.
• Evaluation of the generated reports uses only 100 randomly selected test samples; the test set has 652 samples.
• Dataset is limited to Chinese CSI800 stocks over about two months (Sept-Nov 2024).

Experimental setup

Datasets: FinRpt

Benchmarks: FinRpt

Metrics: CompletionRate Accuracy ROUGE-L BERTScore NumberRate FN News CMI Invest Risk

Baselines: GPT-4o GPT-4o-mini Gemini-2.5-Pro Qwen2.5-72B-Instruct Llama3.1-70B GLM4-9B-Chat XuanYuan-13B-Chat Gemma2-9B FinRobot

Builds on: DAPO GRPO PPO LoRA FinMem FinAgent

Closest papers by meaning

• FinBen: A Holistic Financial Benchmark for Large Language Models (NeurIPS 2024 · similarity 0.89)
• FinSearchComp: Towards a Realistic, Expert-Level Evaluation of Financial Search and Reasoning (ICLR 2026 · similarity 0.88)
• R&D-Agent-Quant: A Multi-Agent Framework for Data-Centric Factors and Model Joint Optimization (NeurIPS 2025 · similarity 0.86)
• EnterpriseBench: Evaluating LLM Agents on End-to-End Spreadsheet Tasks in Finance (NeurIPS 2026 · similarity 0.86)
• FinCon: A Synthesized LLM Multi-Agent System with Conceptual Verbal Reinforcement for Enhanced Financial Decision Making (NeurIPS 2024 · similarity 0.85)
• Fine-Grained Benchmark Generation for Comprehensive Evaluation of Foundation Models (NeurIPS 2026 · similarity 0.85)