Research map / Large language models / AI and society
AncientBench: Towards Comprehensive Evaluation on Excavated and Transmitted Chinese Corpora
Zhihan Zhou, Daqian Shi, Rui Song, Lida Shi, Xiaolei Diao, Hao Xu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune on ancient Chinese knowledge (ablation) | Yi1.5-9B-Chat · AncientBench average (zero-shot) | accuracy: 48.81 -> 50.28 | Improved |
| fine-tune on ancient Chinese knowledge (ablation) | Yi1.5-9B-Chat · Glyph comprehension (zero-shot) | accuracy: 42.70 -> 50.77 | Improved |
| fine-tune on ancient Chinese knowledge (ablation) | Yi1.5-9B-Chat · Contextual comprehension (zero-shot) | accuracy: 56.91 -> 50.93 | Hurt |
| fine-tune on ancient Chinese knowledge (ablation) | Yi1.5-9B-Chat · AncientBench average (few-shot) | accuracy: 50.80 -> 50.83 | No clear effect |
| scale Qwen from 7B to 14B (ablation) | Qwen-Chat · AncientBench average (zero-shot) | accuracy: 43.83 -> 51.00 | Improved |
| scale Baichuan2 from 7B to 13B (ablation) | Baichuan2-Chat · AncientBench average (zero-shot) | accuracy: 48.04 -> 49.29 | Improved |
| add five few-shot examples | Qwen/Baichuan2 models · AncientBench average | accuracy: 51.00 -> 50.05 (Qwen-14B); 49.29 -> 47.92 (Baichuan2-13B) | Hurt |
| add five few-shot examples | Yi1.5-9B-Chat · AncientBench average | accuracy: 48.81 -> 50.80 | Improved |
| add five few-shot examples | GLM4-9b-chat · AncientBench average | accuracy: 49.30 -> 50.53 | Improved |
Closest papers by meaning
- WenMind: A Comprehensive Benchmark for Evaluating Large Language Models in Chinese Classical Literature and Language Arts (NeurIPS 2024 · similarity 0.87)
- Towards Automated Chinese Ancient Character Restoration: A Diffusion-Based Method with a New Dataset (AAAI 2024 · similarity 0.83)
- OBI-Bench: Can LMMs Aid in Study of Ancient Script on Oracle Bones? (ICLR 2025 · similarity 0.83)
- Component-Level Segmentation for Oracle Bone Inscription Decipherment (AAAI 2025 · similarity 0.82)
- Large Language Models' Expert-level Global History Knowledge Benchmark (HiST-LLM) (NeurIPS 2024 · similarity 0.82)
- McHirc: A Multimodal Benchmark for Chinese Idiom Reading Comprehension (AAAI 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
