Research map / Graph learning / Knowledge and temporal graphs
Scalable Feature Learning on Huge Knowledge Graphs for Downstream Machine Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SEPAL, which trains knowledge-graph embeddings (DistMult, etc.) only on a small core of entities and propagates them to the rest of huge graphs via relation-aware message passing, with a new graph splitter BLOCS, to produce better embeddings for downstream feature enrichment at scale.
Problem
Knowledge-graph embedding methods are optimized for link prediction with local negative-sampling contrast, which yields embeddings less useful for downstream ML tasks, and they are hard to scale to huge graphs (tens of millions of entities) without heavy engineering.
Approach
Optimize embeddings only on a small dense core subgraph with a classic KGE model, then propagate them to all outer entities with closed-form, relation-aware message passing (no negative sampling, no gradient descent on outer entities), using overlapping subgraphs from BLOCS so it fits on a single GPU.
Key results
- On link prediction, SEPAL's average MRR (0.5407) is second to PBG (0.6165) and DGL-KE (0.4445) is lower, indicating SEPAL trades link-prediction quality for downstream utility.
- SEPAL achieves the best average downstream performance on each of the 7 source knowledge graphs, and can speed up DistMult more than 20 times for a given configuration.
- SEPAL embeds WikiKG90Mv2 (91M entities, 601M triples) on a single 32GB V100 GPU.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SEPAL instead of local contrastive KGE training | DistMult on downstream feature-enrichment tasks · 46 downstream machine learning tasks | predictive performance | Helped |
| Use SEPAL with DistMult for link prediction | DistMult · Knowledge graph completion across evaluated datasets | MRR: SEPAL average 0.5407; PBG average 0.6165 | Mixed |
| Use SEPAL with DistMult for link prediction | DistMult · Knowledge graph completion across evaluated datasets | Hits@10: SEPAL average 0.6915; PBG average 0.7380 | Mixed |
| Use SEPAL around the base embedding model | DistMult, RotatE, and TransE · Downstream tasks | predictive performance | Helped |
| Use SEPAL instead of standard DistMult training | DistMult · Fixed training configuration | training speed: over 20 times | Helped |
| Use SEPAL instead of PBG, DGL-KE, SMORE, or GraSH | DistMult on Freebase · Four real-world regression tasks | normalized mean cross-validation score | Helped |
Surprising findings
- FastRP, a simple random-projection method that ignores relation types, performs better than most dedicated KGE methods on downstream tasks.
- SEPAL is worse than PBG and DGL-KE on link prediction, supporting the claim that link-prediction performance does not reflect downstream utility.
Limitations
- SEPAL underperforms existing methods (PBG, DGL-KE) on knowledge graph completion because it does not locally contrast positive and negative triples.
- Hybrid core selection with relation-specific edge sampling and reconnection can be computationally expensive for graphs with many relations.
- BLOCS struggles with long chains in graphs like YAGO4.5, and the approach has mainly been validated with DistMult as base model.
Experimental setup
Datasets: YAGO3 YAGO4 YAGO4.5 Freebase WikiKG90Mv2 WikiDBs US elections Housing prices US accidents Movie revenues
Benchmarks: Movie revenues US accidents US elections Housing prices WikiDBs tables link prediction on YAGO3, YAGO4, YAGO4.5, Freebase, WikiKG90Mv2
Metrics: R2 weighted F1 normalized mean score MRR Hits@k MR embedding time
Baselines: DistMult NodePiece PyTorch-BigGraph DGL-KE FastRP SMORE GraSH
Builds on: DistMult REP CompGCN PyTorch-BigGraph FastRP
Closest papers by meaning
- Locally Adaptive Translation for Knowledge Graph Embedding (AAAI 2016 · similarity 0.88)
- SepNE: Bringing Separability to Network Embedding (AAAI 2019 · similarity 0.87)
- You CAN Teach an Old Dog New Tricks! On Training Knowledge Graph Embeddings (ICLR 2020 · similarity 0.86)
- Non-Parametric Estimation of Multiple Embeddings for Link Prediction on Dynamic Knowledge Graphs (AAAI 2017 · similarity 0.86)
- Clustering then Propagation: Select Better Anchors for Knowledge Graph Embedding (NeurIPS 2024 · similarity 0.85)
- Towards Continual Knowledge Graph Embedding via Incremental Distillation (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.