Research map / Large language models / Instruction and fine-tuning
HSKBenchmark: Modeling and Benchmarking Chinese Second Language Acquisition in Large Language Models Through Curriculum Tuning
Qihao Yang, Xuelin Wang, Jiale Chen, Xuelian Dong, Yuxin Hao, Tianyong Hao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add curriculum tuning from HSK3 to HSK6 (staged pretraining plus instruction tuning) | LLaMA2-7B-Chat · HSKBenchmark writing test | holistic score: 70.0000 -> 81.8333 | Improved |
| add curriculum tuning from HSK3 to HSK6 | Mistral-7B-Instruct-v0.3 · HSKBenchmark writing test | holistic score: 76.8333 -> 85.3333 | Improved |
| add curriculum tuning from HSK3 to HSK6 | Chinese-Alpaca-2-7B · HSKBenchmark writing test | holistic score: 77.5000 -> 85.6667 | Improved |
| tune only through HSK3 (first stage) | Chinese-Alpaca-2-7B · HSKBenchmark writing test | holistic score: 77.5000 -> 75.8333 | Hurt |
| add curriculum tuning to reduce writing errors | LLaMA2-7B-Chat · HSKBenchmark writing test | writing errors (Err): 0.9000 -> 0.5333 | Improved |
| add curriculum tuning to raise lexical complexity | Mistral-7B-Instruct-v0.3 · HSKBenchmark writing test | MATTR-50: 0.5260 -> 0.7886 | Improved |
| curriculum tuning versus syntactic complexity on Chinese-Alpaca and Llama2 | Llama2 and Chinese-Alpaca · HSKBenchmark writing test | MDD: 2.4253 -> 2.5558 (Llama2HSK6) | No clear effect |
| order data by HSK level (curriculum) instead of shuffling all data into four stages (ablation) | Llama2, Chinese-Alpaca, Mistral · HSKBenchmark writing test, average score | holistic score | Mixed |
| add curriculum tuning and check general ability after tuning | Llama2 · MMLU | accuracy | No clear effect |
| add curriculum tuning and check Chinese general ability | Llama2 · C-Eval | accuracy | Improved |
Closest papers by meaning
- CK12: A Rounded K12 Knowledge Graph Based Benchmark for Chinese Holistic Cognition Evaluation (AAAI 2024 · similarity 0.88)
- XIFBench: Evaluating Large Language Models on Multilingual Instruction Following (NeurIPS 2025 · similarity 0.88)
- ScholarGEC: Enhancing Controllability of Large Language Model for Chinese Academic Grammatical Error Correction (AAAI 2025 · similarity 0.88)
- Evaluating Large Language Models at Evaluating Instruction Following (ICLR 2024 · similarity 0.88)
- MCIF: Multimodal Crosslingual Instruction-Following Benchmark from Scientific Talks (ICLR 2026 · similarity 0.88)
- WenMind: A Comprehensive Benchmark for Evaluating Large Language Models in Chinese Classical Literature and Language Arts (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
