Research map / Continual learning / Continual fine-tuning of LLMs
Forget by Uncertainty: Orthogonal Entropy Unlearning for Quantized Neural Networks
Tian Zhang, Yujia Tong, Junhao Dong, Ke Xu, Yuze Wang, Jingling Yuan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add gradient orthogonal projection (ablation) | OEU · CIFAR-100 random forgetting 30% (ResNet-18 4-bit) | Average Gap: 3.53 -> 1.98 | Improved |
| add entropy-guided unlearning (vs wrong-label learning) (ablation) | OEU · CIFAR-100 random forgetting 30% (ResNet-18 4-bit) | Average Gap: 3.98 -> 1.98 | Improved |
| use layer-wise instead of global projection (ablation) | OEU · CIFAR-100 random forgetting 30% (ResNet-18 4-bit) | Average Gap: 5.40 -> 1.98 | Improved |
| OEU vs Q-MUL | OEU vs Q-MUL · CIFAR-10 random forgetting 10% (ResNet-18 4-bit) | Average Gap: 1.30 -> 0.58 | Improved |
| OEU vs Q-MUL | OEU vs Q-MUL · CIFAR-100 random forgetting 10% (ResNet-18 4-bit) | Average Gap: 3.11 -> 3.08 | No clear effect |
| OEU vs Q-MUL | OEU vs Q-MUL · CIFAR-100 random forgetting 50% (ResNet-18 4-bit) | Average Gap: 7.06 -> 3.67 | Improved |
| apply OEU to full-precision ResNet-18 | OEU · CIFAR-100 random forgetting 10% | Average Gap: 3.60 | Improved |
| apply OEU with PACT and DSQ quantizers instead of LSQ+ | OEU · CIFAR-100 random forgetting 10% | Average Gap: -60.5% (PACT), -56.2% (DSQ) vs Q-MUL | Improved |
| use gradient ascent/fine-tuning (baselines) vs retrain | FT/GA · CIFAR-100 random forgetting 10% (ResNet-18 4-bit) | Average Gap: 17.43 (FT), 17.69 (GA) | Hurt |
Closest papers by meaning
- Fast Machine Unlearning without Retraining through Selective Synaptic Dampening (AAAI 2024 · similarity 0.88)
- When to Forget? Complexity Trade-offs in Machine Unlearning (ICML 2025 · similarity 0.88)
- Learning What to Forget: Improving LLM Unlearning via Learned Token-Level Importance (NeurIPS 2026 · similarity 0.88)
- Constrained Entropic Unlearning: A Primal-Dual Framework for Large Language Models (NeurIPS 2025 · similarity 0.88)
- Label-Agnostic Forgetting: A Supervision-Free Unlearning in Deep Models (ICLR 2024 · similarity 0.88)
- Certified Unlearning for Neural Networks (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
