Research map / Large language models / Agents, code and math
MCHDoc: A Comprehensive Benchmark for Reading Multi-Carrier Chinese Historical Documents
Yijun Sheng, Shipeng Zhu, Ruijia Zuo, Na Nie, Hui Xue
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune Qwen2.5-VL-3B on multi-carrier historical data (SFT) (ablation) | Qwen2.5-VL-3B · MCHDoc page-level, AncientBook | 1-NED: 29.55 -> 52.53 | Improved |
| fine-tune Qwen2.5-VL-3B (SFT) (ablation) | Qwen2.5-VL-3B · MCHDoc page-level, Inscription | 1-NED: 48.96 -> 48.44 | No clear effect |
| LoRA fine-tune InternVL3.5-1B (ablation) | InternVL3.5-1B · MCHDoc character-level, AncientBook | accuracy: 13.6 -> 62.6 | Improved |
| LoRA fine-tune InternVL3.5-1B (ablation) | InternVL3.5-1B · MCHDoc character-level, JianDu | accuracy: 0.3 -> 17.6 | Improved |
| add LLM post-correction with internal knowledge only | Doubao-Seed-1.6-VL output · MCHDoc, Inscription | 1-NED: 65.90 -> 55.81 | Hurt |
| add LLM post-correction with internal knowledge only | Doubao-Seed-1.6-VL output · MCHDoc, AncientBook | 1-NED: 56.52 -> 51.86 | Hurt |
| add external knowledge base retrieval to post-correction | Gemini-2.5-Flash-Think · MCHDoc, AncientBook | 1-NED: 51.86 -> 65.64 | Improved |
| add external knowledge base retrieval to post-correction | Deepseek-V3.1-Think · MCHDoc, Calligraphy | 1-NED: 43.20 -> 72.92 | Improved |
| add external knowledge base retrieval to post-correction | Gemini-2.5-Flash-Think · MCHDoc, Inscription | 1-NED: 55.81 -> 42.51 | Hurt |
| add external retrieval to post-correction on JianDu | Doubao-Seed-1.6-VL output · MCHDoc, JianDu | 1-NED: 18.82 -> 17.91 (best, Deepseek-V3.1-Think) | No clear effect |
| use GPT series for long historical pages | GPT-5 · MCHDoc page-level, AncientBook | 1-NED: 56.52 -> 6.87 (vs Doubao) | Hurt |
Closest papers by meaning
- Reading between the Lines: Image-Based Order Detection in OCR for Chinese Historical Documents (AAAI 2024 · similarity 0.84)
- Large Language Models' Expert-level Global History Knowledge Benchmark (HiST-LLM) (NeurIPS 2024 · similarity 0.83)
- WenMind: A Comprehensive Benchmark for Evaluating Large Language Models in Chinese Classical Literature and Language Arts (NeurIPS 2024 · similarity 0.82)
- MosaicDoc: A Large-Scale Bilingual Benchmark for Visually Rich Document Understanding (AAAI 2026 · similarity 0.81)
- Towards Automated Chinese Ancient Character Restoration: A Diffusion-Based Method with a New Dataset (AAAI 2024 · similarity 0.81)
- Predicting the Original Appearance of Damaged Historical Documents (AAAI 2025 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
