Research map / Graph learning / Knowledge and temporal graphs

Toward a Unified Statistical Theory of Unsupervised Pretraining and Supervised Neural Knowledge Graph Learning

NeurIPS 2026 · Graph learning › Knowledge and temporal graphs · approach: Knowledge graph embeddings · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes PNKG, a two-stage pipeline (multi-view kernel PCA on entity side information, then relation-wise bounded ReLU networks on labeled triples) with an end-to-end nonasymptotic risk bound decomposing approximation, estimation, pretraining, and optimization error, validated on simulations and WordNet/PrimeKG.

Problem

Scarce labeled triples per relation in knowledge graphs limit expressive KG models, and existing side-information methods lack theoretical grounding for how unlabeled pretraining error affects downstream KG prediction risk.

Approach

Aggregate view-specific entity kernels (including encoder-induced kernels from frozen language models) into a weighted multi-view kernel, take its top eigenvectors as frozen entity coordinates, and train relation-wise ReLU networks on them; a Davis-Kahan-type subspace bound feeds into an end-to-end oracle inequality with signal-adjusted inverse-variance view weighting.

Key results

• On WordNet, uniform multi-view PNKG reaches 0.9508 AUROC, 0.8618 accuracy, 0.8500 F1, improving over the best single view (BGE) by 0.0028 AUROC.
• On PrimeKG hard-relation subset, validation-selected weights (8:1:1) reach 0.8223 AUROC, improving over PubMedBERT by 0.0086 AUROC.
• On PrimeKG overall test set, RotatE is the strongest method; the selected-weight multi-view PNKG improves over PubMedBERT by 0.0043 AUROC but does not beat RotatE.

Surprising findings

• On the full PrimeKG test set, graph-only RotatE outperforms all side-information PNKG variants, which the authors attribute to fixed coordinates on relations with dense structural signal.
• Anisotropic noise increases finite-sample error in the linear-kernel setting but does not create a persistent population bias.

Limitations

• Frozen-coordinate assumption leaves end-to-end bound for fine-tuned embeddings open.
• Shared-subspace condition may hold only approximately for complementary views; i.i.d. triple sampling excludes entity-inductive settings.
• View-weighting rules depend on unknown kernel parameters, requiring estimation.

Experimental setup

Datasets: WordNet PrimeKG

Benchmarks: WordNet PrimeKG

Metrics: AUROC accuracy F1 MSE

Baselines: TransE DistMult ComplEx RotatE random fixed embeddings

Builds on: TransE DistMult ComplEx RotatE Davis-Kahan theorem

Closest papers by meaning

• Interpreting Knowledge Graph Relation Representation from Word Embeddings (ICLR 2021 · similarity 0.89)
• Flock: A Knowledge Graph Foundation Model via Learning on Random Walks (ICLR 2026 · similarity 0.89)
• Towards Foundation Models for Knowledge Graph Reasoning (ICLR 2024 · similarity 0.89)
• Probability Calibration for Knowledge Graph Embedding Models (ICLR 2020 · similarity 0.88)
• Towards Global-Topology Relation Graph for Inductive Knowledge Graph Completion (AAAI 2025 · similarity 0.88)
• Talk like a Graph: Encoding Graphs for Large Language Models (ICLR 2024 · similarity 0.87)