Research map / Large language models / Retrieval and knowledge
From Chaos to Clarity: A Knowledge Graph-Driven Audit Dataset Generation Framework for LLM Unlearning
Weipeng Jiang, Juan Zhai, Shiqing Ma, Ziyan Lei, Xiaofei Xie, Yige Wang, Chao Shen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| audit with LUCID KG-based suite instead of MUSE's 100 questions | unlearned LLaMA2-7B (no unlearning) · MUSE News | KMCs by ROUGE: 33 -> 4688 | Improved |
| audit with LUCID instead of MUSE, entailment criterion | LLaMA2-7B on Books · MUSE Books | KMCs by Entailment: 15 -> 38388 | Improved |
| audit GA+KLR unlearned model with LUCID instead of MUSE | GA+KLR on News · MUSE News | KMCs by ROUGE: 18 -> 3702 | Improved |
| add redundancy removal between forget and retain KGs (ablation) | LUCID on News (GA+KLR) · MUSE News | KMCs by ROUGE: 13298 -> 3702 | Improved |
| add redundancy removal (ablation) | LUCID on News · MUSE News | KMCs by Entailment: 52309 -> 21650 | Improved |
| add redundancy removal to the audit set (ablation) | LUCID on News · MUSE News | dataset-level ROUGE score: 19.7% to 26.1% inflation without dedup | Improved |
| add redundancy removal to the audit set (ablation) | LUCID on News · MUSE News | dataset-level Entailment score: 32.4% to 35.2% inflation without dedup | Improved |
| redundancy removal on forget KG (ablation) | LUCID KG · MUSE News | KG edges: 24,763 -> 16,912 | Improved |
| redundancy removal on MUSE-covered edges | MUSE audit questions · MUSE Books | covered edges: 2,922 -> 473 | Hurt |
| redundancy removal on MUSE-covered edges | MUSE audit questions · MUSE News | covered edges: 193 -> 102 | Hurt |
| apply LUCID to NPO+GDR-unlearned model | NPO+GDR on Books · MUSE Books | KMCs by ROUGE: 0 -> 1435 | Improved |
| KG extraction on narrative Books text vs News | REBEL-based KG extraction · manual assessment | Accuracy of Knowledge Fact: 0.76 -> 0.61 | Hurt |
Closest papers by meaning
- Probing Hidden Knowledge Holes in Unlearned LLMs (NeurIPS 2025 · similarity 0.86)
- Towards Effective Evaluations and Comparisons for LLM Unlearning Methods (ICLR 2025 · similarity 0.85)
- Do LLMs Really Forget? Evaluating Unlearning with Knowledge Correlation and Confidence Awareness (NeurIPS 2025 · similarity 0.85)
- OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metrics (NeurIPS 2025 · similarity 0.84)
- Catastrophic Failure of LLM Unlearning via Quantization (ICLR 2025 · similarity 0.84)
- MUSE: Machine Unlearning Six-Way Evaluation for Language Models (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
