Research map / Large language models / Language, safety and interpretability
Native Chinese Reader: A Dataset Towards Native-Level Chinese Machine Reading Comprehension
Shusheng Xu, Yichen Liu, Xiaoyu Yi, Siyuan Zhou, Huizi Li, Yi Wu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add primary-school Chinese exam data (6000 docs, 13K questions) to training | MacBERT-large · NCR test | accuracy: 0.4780 -> 0.5021 | Improved |
| Competition model (XLNet + BM25 retrieval) instead of fine-tuned MacBERT-large | MacBERT-large · NCR test | accuracy: 0.5021 -> 0.5985 | Improved |
| Filter classical Chinese documents out of the training set (ablation) | MacBERT-large · NCR test, classical (w/o poetry) | accuracy: 0.3871 -> 0.2970 | Hurt |
| Filter classical Chinese documents out of the training set (ablation) | MacBERT-large · NCR test, poetry | accuracy: 0.3582 -> 0.2836 | Hurt |
| Filter classical Chinese documents out of the training set (ablation) | MacBERT-large · NCR test, modern | accuracy: 0.5039 -> 0.5060 | No clear effect |
| Larger/better pretrained model: MacBERT-large vs BERT-Chinese | BERT-based MRC · NCR test | accuracy: 0.3946 -> 0.4780 | Improved |
| Pretrained model vs random guess | BERT-based MRC · NCR test | accuracy: 0.2511 -> 0.3946 | Improved |
| Classical Chinese poetry documents vs modern documents | Best fine-tuned model · NCR test | accuracy: 0.5257 -> 0.3462 | Hurt |
| Classical poetry vs modern documents for humans | Human · NCR test | accuracy: 0.7489 -> 0.9167 | Improved |
| Competition retrieval-based model vs fine-tuned model on sentiment questions | Competition model · NCR test, sentiment questions | accuracy: 0.5641 -> 0.5128 | Hurt |
| Competition model vs fine-tuned model on reasoning questions | Competition model · NCR test, reasoning questions | accuracy: 0.6000 -> 0.6588 | Improved |
| Surface pattern heuristic (combination of absolute quantifiers) for answer prediction | Heuristic · NCR whole dataset | accuracy: 0.2994 | No clear effect |
Closest papers by meaning
- CK12: A Rounded K12 Knowledge Graph Based Benchmark for Chinese Holistic Cognition Evaluation (AAAI 2024 · similarity 0.85)
- McHirc: A Multimodal Benchmark for Chinese Idiom Reading Comprehension (AAAI 2025 · similarity 0.84)
- WenMind: A Comprehensive Benchmark for Evaluating Large Language Models in Chinese Classical Literature and Language Arts (NeurIPS 2024 · similarity 0.84)
- BMMR: A Large-Scale Bilingual Multimodal Multi-Discipline Reasoning Dataset (NeurIPS 2025 · similarity 0.83)
- NovelQA: Benchmarking Question Answering on Documents Exceeding 200K Tokens (ICLR 2025 · similarity 0.83)
- TCM-Ladder: A Benchmark for Multimodal Question Answering on Traditional Chinese Medicine (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
