Research map / Architectures and efficiency / RNNs and normalization

ALRMR-GEC: Adjusting Learning Rate Based on Memory Rate to Optimize the Edit Scorer for Grammatical Error Correction

AAAI 2025 · Architectures and efficiency › RNNs and normalization · approach: Neural scaling laws · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ALRMR-GEC, which adjusts the learning rate of an edit-based GEC scorer using a "memory rate" metric (accuracy on samples newly learned correctly that are excluded from later training), to improve generalization and training speed.

Problem

Gradient-based optimizers for training edit-based grammatical error correction scorers minimize training loss but do not ensure generalization, and the learning rate must be tuned manually, which is costly.

Approach

A memory rate metric, computed as accuracy on a held-out subset of training examples that flipped from incorrect to correct in the previous-to-current epoch and are excluded from subsequent training, serves as a signal to adaptively adjust the learning rate (fast-start backtracking/binary search, then slow adjustment), keeping training near a favorable learning rate.

Key results

• On Roberta-base with threshold 0.5 under Faster Simultaneous Decoding, ALRMR-GEC raises accuracy by about 6.9% (F 49.40 to 56.18 relative to the original editscorer reported in the table row pairing).
• Optimized Roberta-base reaches 84% validation accuracy in 37 epochs (~15 hours) where the 1e-8 baseline remains far from satisfactory after 400 epochs.
• ALRMR-GEC reaches a certain validation accuracy up to tenfold faster in certain cases and yields stable performance across initial learning rates.

Measured effects reported in the paper

Surprising findings

• Gains were smaller for Roberta-large than for Roberta-base under Faster Simultaneous Decoding, which the authors attribute to smaller models being more sensitive to learning rate choice.

Limitations

• Gains on Roberta-large are less pronounced than on Roberta-base.
• Memory rate is defined via accuracy, so the method may favor accuracy over other metrics such as precision.
• Evaluated only on BEA-2019 with edit scorers built from GECToR-style edits.

Experimental setup

Datasets: BEA-2019

Benchmarks: BEA-2019

Metrics: accuracy precision recall F0.5

Baselines: Roberta-base original editscorer Roberta-large original editscorer Pie bea-gector Clang large ft2

Builds on: GECToR AdamW Edit scorer (Sorokin 2022)

Closest papers by meaning

• ScholarGEC: Enhancing Controllability of Large Language Model for Chinese Academic Grammatical Error Correction (AAAI 2025 · similarity 0.89)
• DSGram: Dynamic Weighting Sub-Metrics for Grammatical Error Correction in the Era of Large Language Models (AAAI 2025 · similarity 0.88)
• EXCGEC: A Benchmark for Edit-Wise Explainable Chinese Grammatical Error Correction (AAAI 2025 · similarity 0.86)
• UniEdit: A Unified Knowledge Editing Benchmark for Large Language Models (NeurIPS 2025 · similarity 0.84)
• Wasserstein Distance Constraint and Parameter Sparsification for Batched and Iterative Knowledge Editing (AAAI 2025 · similarity 0.84)
• Golden Layers and Where to Find Them: Improved Knowledge Editing for Large Language Models Via Layer Gradient Analysis (NeurIPS 2026 · similarity 0.83)