Research map / Continual learning / Continual fine-tuning of LLMs
More Edits, More Stable: Understanding the Lifelong Normalization in Sequential Model Editing
Xin Ma, Wei Chen, Qi Liu, Derong Xu, Zhi Zheng, Tong Xu, Enhong Chen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Lifelong Normalization to UltraEdit/RLEdit/MALMEN (ablation) | UltraEdit · Llama-3-8B-Instruct, 20K edits, ZsRE | Efficacy: 36.8 -> 90.1 | Improved |
| add early edits before later edits (positive cumulative effect) (ablation) | LN-based editors · Llama-3-8B-Instruct later segment | Efficacy: 83.9 -> 88.9 (generalization, UltraEdit) | Improved |
| add warm-up stage to StableEdit (ablation) | StableEdit · WikiBigEdit 20K, Mistral-7B-v0.3 | Efficacy: 76.96 -> 78.90 | Improved |
| add full whitening to StableEdit (ablation) | StableEdit · WikiBigEdit 20K, Mistral-7B-v0.3 | Efficacy: 75.12 -> 78.90 | Improved |
| add LN to StableEdit (ablation) | StableEdit · WikiBigEdit 20K, Mistral-7B-v0.3 | Efficacy: 39.13 -> 78.90 | Improved |
| use out-of-domain MedCF warm-up instead of in-domain warm-up (ablation) | StableEdit · ZsRE/FEVER 20K, Mistral-7B-v0.3 | Efficacy/Gen/Spe: average absolute change of only 0.43 points | No clear effect |
| StableEdit instead of UltraEdit at 500K edits | UltraEdit · WikiBigEdit 500K, GPT-J-6B | Generalization: 60.54 -> 65.04 | Improved |
| StableEdit instead of UltraEdit at standard scale | UltraEdit · UltraEditBench 20K, Llama-3-8B-Instruct | Generalization: 81.28 -> 85.46 | Improved |
| StableEdit instead of UltraEdit on specificity | UltraEdit · WikiBigEdit 17K, Llama-3-8B-Instruct | Specificity: 48.51 -> 45.22 | Hurt |
| StableEdit instead of UltraEdit on runtime | UltraEdit · Llama-3-8B-Instruct | seconds per edit: 0.07 -> 0.10 | Hurt |
| remove LN from baselines/editors during update geometry analysis (ablation) | StableEdit and UltraEdit · ZsRE 20K, GPT-J-6B | update cosine similarity | Improved |
Closest papers by meaning
- Towards Lifelong Model Editing via Simulating Ideal Editor (ICML 2025 · similarity 0.94)
- MEMOIR: Lifelong Model Editing with Minimal Overwrite and Informed Retention for LLMs (NeurIPS 2025 · similarity 0.91)
- Perturbation-Restrained Sequential Model Editing (ICLR 2025 · similarity 0.91)
- WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models (NeurIPS 2024 · similarity 0.90)
- ELDER: Enhancing Lifelong Model Editing with Mixture-of-LoRA (AAAI 2025 · similarity 0.90)
- Reinforced Lifelong Editing for Language Models (ICML 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
