Research map / Large language models / Retrieval and knowledge
Integrating Large Language Models and Möbius Group Transformations for Temporal Knowledge Graph Embedding on the Riemann Sphere
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes the 5EL model, which embeds temporal knowledge graph quadruples in projective/Riemann-sphere space using Möbius group transformations and augments them with a QLoRA-fine-tuned LLM that supplies neighbor context for link prediction.
Problem
Temporal knowledge graph embedding models rely on a single geometric space and struggle to jointly model chain, hierarchical, and ring temporal patterns and to use neighboring textual context, especially with sparse link paths.
Approach
Embed TKG entities in complex projective space (CP^d) and model relations as relation-specific Möbius transformations plus temporal rotations, scored by Fubini-Study distance; then fine-tune an LLM with QLoRA and adapter modules to inject neighborhood information into the scoring.
Key results
- 5LE achieves MRR .642 / Hit@1 .571 on ICEWS14 and MRR .713 on ICEWS05-15, outperforming the best listed baselines (e.g., HyIE .631 on ICEWS14).
- On YAGO11k 5LE reaches MRR .217 versus .201 for BiQCap, and on GDELT MRR .282 versus .272 for HyIE.
- Adding the LLM component to existing TKGE models (TeRoL, RotateQVSL, BiQCapL, IMEL, HyIEL) improves each one, per Table 4.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| exclude the LLM component (ablation) | 5LE · ICEWS14 | MRR: .637 -> .642 | Helped |
| exclude the LLM component (ablation) | 5LE · YAGO11k | MRR: .194 -> .217 | Helped |
| exclude the LLM component (ablation) | 5LE · Wikidata12k | MRR: .304 -> .311 | Helped |
| exclude the LLM component (ablation) | 5LE · GDELT | MRR: .264 -> .282 | Helped |
| combine existing TKGE models with the LLM component (ablation) | TERo, RotateQVS, BiQCap, IME, and HyIE · ICEWS14 and YAGO11k | MRR: .562 -> .564; .187 -> .204 (TERo); .591 -> .599; .189 -> .206 (RotateQVS); .592 -> .599; .201 -> .212 (BiQCap); .629 -> | Helped |
| use full-parameter updates instead of QLoRA with PEFT (ablation) | 5LE · ICEWS14 and YAGO11k | Hit@1 | Hurt |
| increase the LoRA adaptation rank (ablation) | 5LE · ICEWS14 and YAGO11k | Hit@1 | Hurt |
| reduce the LoRA adaptation rank (ablation) | 5LE · ICEWS14 and YAGO11k | Hit@1 | Hurt |
| combine full-parameter updates with QLoRA and PEFT (ablation) | 5LE · ICEWS14 and YAGO11k | Hit@1 | Hurt |
Surprising findings
- Full-parameter fine-tuning of the LLM performs worse than QLoRA with adapters, and a hybrid full-plus-LoRA setup also underperforms the baseline.
Limitations
- Ablation shows gains only with a single-step setting where past ground truth is available at test time.
- The specific LLM backbone, its size, and hardware are not clearly specified in the text.
Experimental setup
Datasets: ICEWS14 ICEWS05-15 YAGO11k Wikidata12k GDELT
Benchmarks: ICEWS14 ICEWS05-15 YAGO11k Wikidata12k GDELT
Metrics: MRR Hit@1 Hit@3 Hit@10
Baselines: TTransE TA-TransE HyTE DE-SimplE ATiSE TeRo HyIE IME BiQCap Vicuna GenTKG ChronoR
Builds on: QLoRA LoRA ComplEx RotatE HERCULES
Closest papers by meaning
- Large Language Models-guided Dynamic Adaptation for Temporal Knowledge Graph Reasoning (NeurIPS 2024 · similarity 0.90)
- HGE: Embedding Temporal Knowledge Graphs in a Product Space of Heterogeneous Geometric Subspaces (AAAI 2024 · similarity 0.89)
- TGB 2.0: A Benchmark for Learning on Temporal Knowledge Graphs and Heterogeneous Graphs (NeurIPS 2024 · similarity 0.87)
- Time-Aware Knowledge Representations of Dynamic Objects with Multidimensional Persistence (AAAI 2024 · similarity 0.86)
- DECRL: A Deep Evolutionary Clustering Jointed Temporal Knowledge Graph Representation Learning Approach (NeurIPS 2024 · similarity 0.86)
- Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.