Research map / Vision-language and multimodal / Multimodal LLMs
FinMMR: Make Financial Numerical Reasoning More Multimodal, Comprehensive, and Challenging
Zichen Tang, Haihong E, Jiacheng Liu, Zhongjun Yang, Rongjin Li, Zihua Rong, Haoyang He, Zhuodi Hao, Xinyang Hu, Kun Ji, Ziyan Ma, Mengyuan Ji and 9 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| enable 64K extended thinking (ablation) | Claude 3.7 Sonnet · FinMMR test Hard (CoT) | accuracy (%): 50.80 -> 53.00 | Improved |
| enable 64K extended thinking (token cost) (ablation) | Claude 3.7 Sonnet · FinMMR test Hard tokens | tokens (M): 0.34M -> 4.06M | Hurt |
| use PoT prompting instead of CoT (mean over models excluding QVQ) (ablation) | MLLMs · FinMMR test Hard | accuracy (%): 36.20 -> 37.64 | Improved |
| use PoT instead of CoT (ablation) | Qwen2.5-VL-72B · FinMMR test Medium | accuracy (%): 63.42 -> 64.17 | Improved |
| use PoT instead of CoT (ablation) | QVQ-72B-Preview · FinMMR test Hard | accuracy (%): 40.30 -> 6.20 | Hurt |
| use PoT instead of CoT (ablation) | GPT-4o · FinMMR test Hard | accuracy (%): 45.40 -> 47.80 | Improved |
| add distractor images to ground images (ablation) | Qwen2.5-VL-72B · FinMMR Medium validation (PoT) | accuracy (%): 77.78 -> 64.73 | Hurt |
| add distractor images to ground images (ablation) | Qwen2.5-VL-72B · FinMMR test Hard (PoT) | accuracy (%): 57.18 -> 47.23 | Hurt |
| add visual filtering stage before reasoning (ablation) | Qwen2.5-VL-72B · FinMMR Medium validation (PoT, distractors) | accuracy (%): 64.73 -> 71.56 | Improved |
| add MLLM-instructed knowledge retrieval and judging (financial function library) (ablation) | Gemini 2.0 Flash Thinking · FinMMR tabular QA (1,160 instances, PoT) | accuracy (%): 78.71 -> 83.02 | Improved |
| add knowledge augmentation (ablation) | GPT-4o · FinMMR tabular QA (1,160 instances, PoT) | accuracy (%): 80.60 -> 83.62 | Improved |
| add knowledge augmentation (ablation) | Claude 3.7 Sonnet (64K thinking) · FinMMR tabular QA (1,160 instances, PoT) | accuracy (%): 83.53 -> 86.29 | Improved |
Closest papers by meaning
- FinMMDocR: Benchmarking Financial Multimodal Reasoning with Scenario Awareness, Document Understanding, and Multi-Step Computation (AAAI 2026 · similarity 0.95)
- FinBen: A Holistic Financial Benchmark for Large Language Models (NeurIPS 2024 · similarity 0.90)
- FinSearchComp: Towards a Realistic, Expert-Level Evaluation of Financial Search and Reasoning (ICLR 2026 · similarity 0.87)
- BMMR: A Large-Scale Bilingual Multimodal Multi-Discipline Reasoning Dataset (NeurIPS 2025 · similarity 0.86)
- STEER-ME: Assessing the Microeconomic Reasoning of Large Language Models (NeurIPS 2025 · similarity 0.86)
- FinCon: A Synthesized LLM Multi-Agent System with Conceptual Verbal Reinforcement for Enhanced Financial Decision Making (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
