Research map / Optimization / Optimizers

MetaKE: Meta-Learning for Knowledge Editing Toward a Better Accuracy-Editability Trade-off

NeurIPS 2026 · Optimization › Optimizers · approach: Gradient-based meta-learning theory · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MetaKE, a bi-level framework that lets upstream target optimization in locate-then-edit knowledge editing receive downstream constraint feedback via a Structural Gradient Proxy, improving the accuracy-editability trade-off.

Problem

Locate-then-edit knowledge editing optimizes the target representation upstream with uniform KL regularization that is blind to how much of the planned residual is realized under downstream constrained parameter updates, hurting the accuracy-editability trade-off.

Approach

Casts upstream target optimization and downstream constrained parameter updates as a bi-level problem where the target is the outer meta-parameter, and approximates the hypergradient through a closed-form Structural Gradient Proxy at the final edited layer instead of differentiating through all layers.

Key results

• On LLaMA3 (8B) ZsRE, MetaKE reaches 96.84 Efficacy, 92.45 Generalization and 33.12 Specificity, above AlphaEdit-BLUE and AlphaEdit.
• On ZsRE with GPT2-XL, MetaKE improves Generalization by 9.10% relative to AlphaEdit (per the text).
• Ablation on LLaMA3/ZsRE: removing bi-level optimization drops Efficacy from 96.84 to 94.27 and Generalization from 92.45 to 88.35.

Limitations

• Theory relies on a local quadratic surrogate and single-layer approximation; nonlinear effects are treated as higher-order residuals.
• Structural Gate instantiated only for the AlphaEdit-style projected update; extension to other downstream editors is left open.
• Gains over the strongest baselines are modest on several metrics (e.g., Specificity on GPT-J ZsRE).

Experimental setup

Datasets: ZsRE CounterFact

Benchmarks: ZsRE CounterFact GLUE (SST, MRPC, CoLA, RTE, NLI) MMLU

Metrics: Efficacy Generalization Specificity Fluency Consistency

Baselines: MEND MALMEN ROME MEMIT PRUNE RECT PMET AlphaEdit BLUE SPHERE

Builds on: MEMIT AlphaEdit PMET ROME MEND

Closest papers by meaning

• AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models (ICLR 2025 · similarity 0.88)
• Uncovering Overfitting in Large Language Model Editing (ICLR 2025 · similarity 0.87)
• Wasserstein Distance Constraint and Parameter Sparsification for Batched and Iterative Knowledge Editing (AAAI 2025 · similarity 0.87)
• UniEdit: A Unified Knowledge Editing Benchmark for Large Language Models (NeurIPS 2025 · similarity 0.87)
• In-Context Editing: Learning Knowledge from Self-Induced Distributions (ICLR 2025 · similarity 0.87)
• Golden Layers and Where to Find Them: Improved Knowledge Editing for Large Language Models Via Layer Gradient Analysis (NeurIPS 2026 · similarity 0.86)