Research map / Large language models / Language, safety and interpretability
JELV: A Judge of Edit-Level Validity for Evaluation and Automated Reference Expansion in Grammatical Error Correction
Yuhao Zhan, Yuqing Zhang, Jing Yuan, Qixiang Ma, Zhiqi Yang, Yu Gu, Zemin Liu, Fei Wu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| chain DS-V3, GPT-4o and Qwen-Max with each reviewing the previous judgment (ablation) | JELV1.0 · PEVData | accuracy: 0.8415 -> 0.8976 -> 0.9134 | Improved |
| add explain-then-annotate, in-context learning and context-aware prompting (ablation) | JELV1.0 · PEVData | accuracy: exceeds 90% combined | Improved |
| reclassify false positives with JELV before scoring (ablation) | JELV-based F(x) · SEEDA sentence-level meta-evaluation | correlation | Improved |
| add false positive decoupling (overcorrection weight alpha) (ablation) | JELV-based F(x) · SEEDA sentence-level meta-evaluation | correlation | Improved |
| add GPT-2 fluency score to the metric (ablation) | JELV-based F(x) · SEEDA sentence-level meta-evaluation | correlation | Improved |
| JELV-based F(x) versus LLM-based metric | metric vs human judgment · SEEDA social media (system-level SEEDA-E Base) | Pearson r: 0.951 -> 0.975 | Improved |
| JELV-based F(x) versus SOME on sentence-level | metric vs human judgment · SEEDA social media (SEEDA-E) | accuracy: 0.394 -> 0.780 | Improved |
| retrain on BEA19 expanded with LLM candidates, unfiltered | Ensembles GEC system · CoNLL14 (M2) | M2: 72.68 -> 72.98 | Improved |
| filter LLM candidates with JELV2.0 before adding to training data (ablation) | Ensembles GEC system · CoNLL14 (M2) | M2: 72.98 -> 73.05 | Improved |
| filter LLM candidates with JELV2.0 (ablation) | Sequence tagging GEC system · CoNLL14 (JELV-based F(x)) | F(x): 75.70 -> 76.95 | Improved |
| retrain on JELV-filtered expanded BEA19 | Ensembles GEC system · CoNLL14 (JELV-based F(x)) | F(x): 79.47 -> 80.45 | Improved |
| retrain on unfiltered expanded BEA19 | Majority-voting GEC system · CoNLL14 (M2) | M2: 71.44 -> 71.44 | No clear effect |
Closest papers by meaning
- DSGram: Dynamic Weighting Sub-Metrics for Grammatical Error Correction in the Era of Large Language Models (AAAI 2025 · similarity 0.91)
- ScholarGEC: Enhancing Controllability of Large Language Model for Chinese Academic Grammatical Error Correction (AAAI 2025 · similarity 0.89)
- ALRMR-GEC: Adjusting Learning Rate Based on Memory Rate to Optimize the Edit Scorer for Grammatical Error Correction (AAAI 2025 · similarity 0.89)
- EXCGEC: A Benchmark for Edit-Wise Explainable Chinese Grammatical Error Correction (AAAI 2025 · similarity 0.87)
- RevisEval: Improving LLM-as-a-Judge via Response-Adapted References (ICLR 2025 · similarity 0.85)
- UniEdit: A Unified Knowledge Editing Benchmark for Large Language Models (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
