Research map / Continual learning / Online and replay-based
Towards Continuous Reuse of Graph Models via Holistic Memory Diversification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DMSG, a continual graph learning method that selects intra- and inter-class diverse memory nodes via greedy selection and replays variational, adversarially regularized, graph-reconstruction-consolidated synthetic node embeddings.
Problem
Class-incremental learning on growing graphs where new nodes bring novel classes; memory replay suffers from low-quality, low-diversity buffer selection and limited buffer knowledge that the model overfits to, causing catastrophic forgetting.
Approach
Select buffer nodes by a greedy submodular-style objective combining intra-class and inter-class distance diversity, then replay synthetic buffer-node embeddings sampled from a variational layer, regularized by an adversarial discriminator, a label-based graph reconstruction decoder, and KL divergence.
Key results
- DMSG reaches 77.8% AA on CoraFull vs 77.7% for SEM and 81.2% for Joint, with AF of -0.5% vs -10.0% for SEM.
- On Reddit DMSG achieves 98.1% AA, exceeding Joint (97.1%) and SEM (96.3%).
- On OGB-Products DMSG reaches 66.0% AA vs 65.1% for SEM; on OGB-Arxiv 50.7% vs 49.9%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use DMSG instead of SEM | incremental graph learning with a 2-layer GCN · CoreFull | AA (%): 77.7$\pm$0.8 -> 77.8$\pm$0.3 | Helped |
| use DMSG instead of SEM | incremental graph learning with a 2-layer GCN · OGB-Arxiv | AA (%): 49.9$\pm$0.6 -> 50.7$\pm$0.4 | Helped |
| use DMSG instead of SEM | incremental graph learning with a 2-layer GCN · Reddit | AA (%): 96.3$\pm$0.1 -> 98.1$\pm$0.0 | Helped |
| use DMSG instead of SEM | incremental graph learning with a 2-layer GCN · OGB-Products | AA (%): 65.1$\pm$1.0 -> 66.0$\pm$0.4 | Helped |
| add adversarial embedding loss (ablation) | DMSG memory replay · CoreFull | AA (%): 74.4$\pm$0.8 -> 77.8$\pm$0.3 | Helped |
| add adversarial embedding loss (ablation) | DMSG memory replay · OGB-Arxiv | AA (%): 49.3$\pm$0.3 -> 50.7$\pm$0.4 | Helped |
| add adversarial embedding loss (ablation) | DMSG memory replay · Reddit | AA (%): 95.1$\pm$2.9 -> 98.1$\pm$0.0 | Helped |
| add adversarial embedding loss (ablation) | DMSG memory replay · OGB-Products | AA (%): 60.1$\pm$0.9 -> 66.0$\pm$0.4 | Helped |
| add graph reconstruction loss (ablation) | DMSG memory replay · CoreFull | AA (%): 74.8$\pm$0.7 -> 77.8$\pm$0.3 | Helped |
| add graph reconstruction loss (ablation) | DMSG memory replay · OGB-Arxiv | AA (%): 49.7$\pm$0.2 -> 50.7$\pm$0.4 | Helped |
| add graph reconstruction loss (ablation) | DMSG memory replay · Reddit | AA (%): 97.8$\pm$0.4 -> 98.1$\pm$0.0 | Helped |
| add graph reconstruction loss (ablation) | DMSG memory replay · OGB-Products | AA (%): 60.5$\pm$0.7 -> 66.0$\pm$0.4 | Helped |
Surprising findings
- DMSG outperforms the Joint upper bound on Reddit and shows positive average forgetting (0.9%).
- On CoraFull, DMSG's AA (77.8%) is only marginally above SEM (77.7%), while Joint reaches 81.2%.
Limitations
- Does not improve the graph feature extractor, so capacity may limit performance as graphs grow (acknowledged by authors).
- Gains over SEM are small on CoraFull and OGB-Arxiv, and DMSG remains below Joint on CoraFull and OGB-Arxiv.
- Theoretical justification relies on Gaussian assumptions and an unbiased-sampling assumption; the submodularity lemma argument is informal.
Experimental setup
Datasets: CoraFull OGB-Arxiv Reddit OGB-Products
Benchmarks: CoraFull OGB-Arxiv Reddit OGB-Products
Metrics: Average Accuracy (AA) Average Forgetting (AF)
Baselines: Fine-tune Joint EWC MAS GEM LwF TWP ER-GNN SSM SEM
Builds on: ER-GNN SEM TWP GraphSAGE gradient reversal layer
Closest papers by meaning
- A Topology-aware Graph Coarsening Framework for Continual Graph Learning (NeurIPS 2024 · similarity 0.88)
- Efficient Memory Crystallization for Graph Learning under Non-Stationary Distribution Shifts (NeurIPS 2026 · similarity 0.88)
- What Matters in Graph Class Incremental Learning? An Information Preservation Perspective (NeurIPS 2024 · similarity 0.88)
- GraphKeeper: Graph Domain-Incremental Learning via Knowledge Disentanglement and Preservation (NeurIPS 2025 · similarity 0.87)
- GCAL: Adapting Graph Models to Evolving Domain Shifts (ICML 2025 · similarity 0.87)
- Replay-and-Forget-Free Graph Class-Incremental Learning: A Task Profiling and Prompting Approach (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.