Research map / Continual learning / Online and replay-based
Towards Continual Knowledge Graph Embedding via Incremental Distillation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes IncDE, a continual knowledge graph embedding method that orders new triples hierarchically by graph structure and uses incremental distillation of entity representations plus two-stage training on TransE.
Problem
Continual knowledge graph embedding (CKGE): efficiently learning emerging triples in a growing KG while preserving old knowledge, where existing methods ignore graph structure in learning order and in old-knowledge preservation.
Approach
Layers new triples by BFS distance from the old graph, orders triples within layers by node and betweenness centrality, distills entity representations layer by layer weighted by graph importance, and trains in two stages (freeze old embeddings first, then train all).
Key results
- IncDE achieves MRR 0.253 on ENTITY and 0.234 on GraphEqual, outperforming the best baseline LKGE (0.234 and 0.214).
- Removing incremental distillation drops MRR by 0.2%-6.5% across datasets, the largest contribution among the three components.
- IncDE improves MRR by 1.8%-2.0% over the best baseline on the unequal-growth datasets GraphHigher and GraphLower.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add incremental distillation (ablation) | IncDE · ENTITY, RELATION, FACT, HYBRID, GraphEqual, GraphHigher, and GraphLower | MRR: 0.2\%-6.5\% | Helped |
| add hierarchical ordering (ablation) | IncDE · ENTITY, RELATION, FACT, HYBRID, GraphEqual, GraphHigher, and GraphLower | MRR: 0.2\%-1.8\% | Helped |
| add two-stage training (ablation) | IncDE · ENTITY, RELATION, FACT, HYBRID, GraphEqual, GraphHigher, and GraphLower | MRR: 0.2\%-1.8\% | Helped |
| use IncDE instead of the strongest baseline | TransE-based CKGE · ENTITY, RELATION, FACT, HYBRID, GraphEqual, GraphHigher, and GraphLower | MRR: 1.4\%; 1.8\%-2.0\% | Helped |
Surprising findings
- Removing hierarchical ordering and two-stage training causes only slight drops, while incremental distillation is by far the most important component.
- On RELATION and FACT, IncDE improves MRR by only 0.6%-0.7% over the best baseline.
Limitations
- Does not handle deletion of old knowledge as KGs evolve.
- Does not address integration of cross-domain or heterogeneous data.
- Gains on RELATION and FACT are marginal, attributed to limited entity growth.
Experimental setup
Datasets: ENTITY RELATION FACT HYBRID GraphEqual GraphHigher GraphLower
Benchmarks: ENTITY RELATION FACT HYBRID GraphEqual GraphHigher GraphLower
Metrics: MRR Hits@1 Hits@10
Baselines: Fine-tune PNN CWR GEM EMR DiCGRL SI EWC LKGE
Builds on: TransE knowledge distillation for KGE LKGE EWC
Closest papers by meaning
- GraphKeeper: Graph Domain-Incremental Learning via Knowledge Disentanglement and Preservation (NeurIPS 2025 · similarity 0.88)
- Knowledge Graph Enhanced Generative Multi-modal Models for Class-Incremental Learning (NeurIPS 2025 · similarity 0.88)
- Clustering then Propagation: Select Better Anchors for Knowledge Graph Embedding (NeurIPS 2024 · similarity 0.87)
- KGDM: A Diffusion Model to Capture Multiple Relation Semantics for Knowledge Graph Embedding (AAAI 2024 · similarity 0.87)
- Towards Global-Topology Relation Graph for Inductive Knowledge Graph Completion (AAAI 2025 · similarity 0.86)
- Toward a Unified Statistical Theory of Unsupervised Pretraining and Supervised Neural Knowledge Graph Learning (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.