Research map / Federated learning and privacy / Privacy attacks and secure learning
MUSE: Machine Unlearning Six-Way Evaluation for Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
The paper introduces MUSE, a benchmark for machine unlearning in language models evaluated across six ways (no verbatim/knowledge memorization, privacy leakage, utility preservation, scalability, sustainability, etc.), using news and books corpora.
Problem
Approximate unlearning methods for language models are hard to compare because existing evaluations are narrow; the paper targets removing copyrighted or private training text from trained LMs while preserving utility.
Approach
A comprehensive six-way benchmark (MUSE) that evaluates unlearning algorithms on both forgetting and retention across no verbatim memorization, no knowledge memorization, no privacy leakage, utility preservation, scalability, and sustainability, using news and books data.
Key results
- No single unlearning method satisfies all six MUSE criteria in the reported experiments.
Limitations
- The provided text is only the NeurIPS template and does not contain the paper's actual results, so the details are limited.
Experimental setup
Datasets: news articles books
Benchmarks: MUSE
Metrics: verbatim memorization knowledge memorization privacy leakage utility preservation scalability sustainability
Baselines: gradient ascent task vector NPO RMU
Builds on: gradient ascent unlearning task vector
Closest papers by meaning
- A Closer Look at Machine Unlearning for Large Language Models (ICLR 2025 · similarity 0.94)
- Towards Effective Evaluations and Comparisons for LLM Unlearning Methods (ICLR 2025 · similarity 0.92)
- To Each (Textual Sequence) Its Own: Improving Memorized-Data Unlearning in Large Language Models (ICML 2024 · similarity 0.92)
- OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metrics (NeurIPS 2025 · similarity 0.91)
- On Large Language Model Continual Unlearning (ICLR 2025 · similarity 0.91)
- Catastrophic Failure of LLM Unlearning via Quantization (ICLR 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.