Research map / Large language models / Language, safety and interpretability

ScholarGEC: Enhancing Controllability of Large Language Model for Chinese Academic Grammatical Error Correction

AAAI 2025 · Large language models › Language, safety and interpretability · approach: Multilingual LLM data and culture · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ScholarGEC, a framework that adds disciplinary knowledge prefixes, a detection-correction split with a special no-error token, and iterative self-reflection to fine-tuned LLMs to reduce over-correction in Chinese academic grammatical error correction.

Problem

LLMs over-correct error-free domain-specific terms and expressions when correcting Chinese academic papers, due to limited domain understanding and poor controllability of generative Seq2Seq correction.

Approach

Combines retrieved and LLM-generated disciplinary knowledge prefixes with a two-stage detection-then-correction structure, a special token for error-free sentences, and iterative self-reflection, so the model can skip correction for correct text and better control edits.

Key results

• On AcaCGEC-test, Llama3-8B-Chinese-Chat with ScholarGEC reaches F0.5 31.73 versus 21.05 without the framework (BART baseline 10.75).
• ScholarGEC improves F0.5 for all four LLM backbones on all datasets, with the gain on AcaCGEC about 10 points larger than on other datasets.
• Human evaluation of 20 volunteers shows the ScholarGEC-enhanced Llama3-Chinese achieves higher terminology and content scores than ChatGPT-3.5 and sets a new average score SOTA.

Measured effects reported in the paper

Surprising findings

• Applying the framework to un-finetuned LLMs decreased performance on AcaCGEC-test, suggesting domain-knowledge prefixes can add instability without sufficient domain understanding.
• Qwen2.5-14B sometimes underperforms Llama3-8B on general datasets, despite its larger size.

Limitations

• Only models up to about 14B parameters were tested due to compute constraints.
• Self-reflection prompt engineering was not extensively explored.
• Gains are smaller in general-domain GEC than in academic-domain GEC.

Experimental setup

Datasets: AcaCGEC NaCGEC ECSpell

Benchmarks: AcaCGEC-test NaCGEC-test ECSpell-test

Metrics: precision recall F0.5

Baselines: BART Llama3-8B-Chinese-Chat Baichuan2-7B-Chat Qwen2-7B-Instruct Qwen2.5-14B-Instruct ChatGPT-3.5

Builds on: Self-Refine Seq2Edit detection-correction LoRA Focal loss

Closest papers by meaning

• DSGram: Dynamic Weighting Sub-Metrics for Grammatical Error Correction in the Era of Large Language Models (AAAI 2025 · similarity 0.90)
• EXCGEC: A Benchmark for Edit-Wise Explainable Chinese Grammatical Error Correction (AAAI 2025 · similarity 0.89)
• ALRMR-GEC: Adjusting Learning Rate Based on Memory Rate to Optimize the Edit Scorer for Grammatical Error Correction (AAAI 2025 · similarity 0.89)
• UniEdit: A Unified Knowledge Editing Benchmark for Large Language Models (NeurIPS 2025 · similarity 0.87)
• Gaining Wisdom from Setbacks: Aligning Large Language Models via Mistake Analysis (ICLR 2024 · similarity 0.87)
• Can LLMs Correct Themselves? A Benchmark of Self-Correction in LLMs (NeurIPS 2025 · similarity 0.87)