Research map / Learning theory and algorithms / Fairness
CURE4Rec: A Benchmark for Recommendation Unlearning with Deeper Influence
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces CURE4Rec, a benchmark evaluating recommendation unlearning methods on completeness, utility, efficiency, and fairness under core, edge, and random unlearning sets across ML-100K, ML-1M, and Amazon Digital Music.
Problem
Evaluating recommendation unlearning lacks a unified framework and overlooks deeper influences such as fairness and robustness to different unlearning sets.
Approach
A four-aspect benchmark (completeness via membership inference oracle, utility via NDCG/HR, efficiency via runtime, and group/shard fairness) that varies the unlearning set by graph-based node importance (core, edge, random) for collaborative filtering recommenders.
Key results
- SCIF (approximate) achieves utility closest to Retrain on ML-100K (NDCG@20 about 0.27-0.29 vs Retrain 0.287-0.319 under WMF) and is much faster than exact methods (e.g., 289-325 s vs 4296-4526 s Retrain on ML-100K WMF).
- Exact methods (SISA, RecEraser, UltraRE) fully unlearn by design but worsen active/inactive group fairness, with RecEraser and UltraRE clustering active users into shards.
- SCIF shows MIO accuracy about 4.1% higher than Retrain on average, with weakest unlearning on edge data.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SCIF instead of exact-unlearning methods | Recommendation unlearning · ML-100K, ML-1M, and ADM; random unlearning set; WMF | NDCG@20: 0.2092 -> 0.2699; 0.1891 -> 0.1922; 0.3334 -> 0.0227 | Mixed |
| Use SCIF instead of SISA | Recommendation unlearning · ML-100K (WMF) | unlearning time (s): 467 -> 325 | Helped |
| Use SCIF instead of Retrain | Recommendation unlearning · ML-100K (WMF) | unlearning completeness (MIO accuracy; closer to 0.5 is better): 0.538 -> 0.563 | Hurt |
| Select core users rather than random users for unlearning | Exact unlearning methods · Across datasets and models | recommendation utility | Helped |
| Select edge data rather than core or random data for SCIF | SCIF · Across datasets | unlearning completeness (MIO accuracy; closer to 0.5 is better) | Hurt |
| Use RecEraser or UltraRE rather than SISA | Exact recommendation unlearning · Across datasets and models, same unlearning set | recommendation utility | Helped |
| Increase the number of shards | Exact unlearning · WMF on ML-100K | unlearning efficiency | Helped |
| Increase the number of shards | Exact unlearning · WMF on ML-100K | recommendation utility | Hurt |
| Increase the number of shards | Exact unlearning · WMF on ML-100K | group-level fairness | Hurt |
| Increase the unlearning data ratio | Recommendation unlearning · WMF on ML-100K | recommendation utility | Hurt |
Surprising findings
- Approximate unlearning (SCIF) outperforms exact methods in utility, efficiency, and fairness, but only achieves weak unlearning completeness.
- Clustering of active users by RecEraser and UltraRE amplifies group unfairness relative to SISA's random assignment.
Limitations
- Metrics within each aspect have considerable room for improvement, as stated by the authors.
- Only three collaborative filtering models and user-wise unlearning are studied.
- No error bars reported (checklist answered No).
Experimental setup
Datasets: ML-100K ML-1M ADM
Benchmarks: ML-100K ML-1M ADM
Metrics: NDCG@20 HR@20 MIO accuracy A-IGF shardGF running time (s)
Baselines: Retrain SISA RecEraser UltraRE SCIF
Builds on: SISA RecEraser UltraRE SCIF
Closest papers by meaning
- RRL: Recommendation Reverse Learning (AAAI 2024 · similarity 0.88)
- Multi-Modal Recommendation Unlearning for Legal, Licensing, and Modality Constraints (AAAI 2025 · similarity 0.87)
- Tree of Preferences for Diversified Recommendation (NeurIPS 2025 · similarity 0.86)
- Influential Exemplar Replay for Incremental Learning in Recommender Systems (AAAI 2024 · similarity 0.85)
- RecFlow: An Industrial Full Flow Recommendation Dataset (ICLR 2025 · similarity 0.85)
- Erase Then Rectify: A Training-Free Parameter Editing Approach for Cost-Effective Graph Unlearning (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.