Research map / Large language models / Language, safety and interpretability

EXCGEC: A Benchmark for Edit-Wise Explainable Chinese Grammatical Error Correction

AAAI 2025 · Large language models › Language, safety and interpretability · approach: Controlled text generation · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces Explainable Chinese GEC (EXGEC), a 8,216-sample edit-wise explanation benchmark (EXCGEC), and benchmarks post- vs pre-explaining multi-task LLMs with a COTE decoding algorithm.

Problem

Grammatical error correction systems rarely explain their corrections, and prior explanation work treats correction and explanation separately with no comprehensive Chinese benchmark combining both.

Approach

Jointly train LLMs to output corrections and edit-wise explanations (error type, severity, free-text description) in either post-explaining (correct-then-explain) or pre-explaining order, and decode post-explaining models with COTE, which extracts edits from the generated correction before explaining.

Key results

• Post-explaining Qwen1.5-7B-chat reaches CLEME F0.5 26.54 and ChERRANT F0.5 29.98, outperforming pre-explaining Qwen1.5-7B-chat (13.69 and 16.22).
• A GEC-GEE pipeline built on Qwen1.5-7B-chat outperforms multi-task post-explaining models, scoring 30.29 CLEME F0.5 versus 26.54.
• Human correlations show ROUGE-1 has the highest Pearson correlation with human ratings of error descriptions (0.9464), followed by METEOR (0.9280).

Measured effects reported in the paper

Surprising findings

• The multi-task models underperform a simple GEC-GEE pipeline, suggesting difficulty learning both tasks jointly at 7B scale.
• Top-p decoding for correction causes over-correction, with precision dropping sharply compared with beam search.

Limitations

• Explanation data is largely synthesized by GPT-4 and then filtered, so explanations may carry GPT-4 biases and failure modes.
• Multi-task models at 7B scale underperform the pipeline, indicating limited capacity to learn both tasks jointly.
• Human evaluation uses only 2 annotators on 100 samples.

Experimental setup

Datasets: FCGEC YACLC MuCGEC NaCGEC NLPCC HSK

Benchmarks: EXCGEC-test

Metrics: CLEME P/R/F0.5 ChERRANT P/R/F0.5 Hit rate Miss rate error type Acc error type macro-F1 severity MAE BLEU METEOR ROUGE-1/2/L

Baselines: GEC-GEE pipeline Qwen1.5-7B-chat pre-explaining Llama3-8B-instruct DeepSeek-7B-chat

Builds on: ERRANT CLEME LLaMA-Factory Rationalization Prompting

Closest papers by meaning

• ScholarGEC: Enhancing Controllability of Large Language Model for Chinese Academic Grammatical Error Correction (AAAI 2025 · similarity 0.89)
• ALRMR-GEC: Adjusting Learning Rate Based on Memory Rate to Optimize the Edit Scorer for Grammatical Error Correction (AAAI 2025 · similarity 0.86)
• DSGram: Dynamic Weighting Sub-Metrics for Grammatical Error Correction in the Era of Large Language Models (AAAI 2025 · similarity 0.85)
• Automated Assessment of Fidelity and Interpretability: An Evaluation Framework for Large Language Models’ Explanations (Student Abstract) (AAAI 2024 · similarity 0.85)
• UniEdit: A Unified Knowledge Editing Benchmark for Large Language Models (NeurIPS 2025 · similarity 0.83)
• Should We Really Edit Language Models? On the Evaluation of Edited Language Models (NeurIPS 2024 · similarity 0.82)