Research map / Large language models / Retrieval and knowledge
MKGL: Mastery of a Three-Word Language
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces KG Language (KGL), a three-word entity-relation-entity sentence format, and MKGL, which fine-tunes Llama-2-7b with LoRA-based context and score retrievers to create KGL token embeddings for KG completion, reporting gains over embedding and LLM baselines on FB15k-237 and WN18RR.
Problem
LLMs struggle to handle KG entities and relations, which are atomic identifiers absent from their vocabulary, so scaling LLM-based KG completion to large entity sets (thousands of candidates) remains inefficient and limited to simple tasks like triplet classification.
Approach
Treat KG facts as a constrained three-word language (KGL) and create new LLM token embeddings for KG entities and relations by aggregating their textual token embeddings and KG neighborhood via LoRA-style down/up-scaled PNA encoders, plus a score retriever that outputs one-shot probabilities over all candidate entities.
Key results
- On FB15k-237, MKGL reaches MRR .415 and Hits@10 .591, versus KICGPT's .412 and .554.
- On WN18RR, MKGL reaches MRR .552 and Hits@1 .500, outperforming KICGPT (.549, .474).
- On inductive FB15k-237-ind-v1, MKGL reaches MRR .475 versus RED-GNN's .369.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MKGL instead of KICGPT | KG completion · FB15k-237 | MRR: .412 -> .415 | Helped |
| Use MKGL instead of KICGPT | KG completion · WN18RR | MRR: .549 -> .552 | Helped |
| Use MKGL instead of RED-GNN | inductive KG completion · FB15k-237-ind-v1 | MRR: .369 -> .475 | Helped |
| Use MKGL instead of RED-GNN | inductive KG completion · WN18RR-ind-v1 | MRR: .701 -> .746 | Helped |
| Add text retrieval to the score retriever (ablation) | MKGL · FB15k-237 | MRR: .382 -> .415 | Helped |
| Add text retrieval to the score retriever (ablation) | MKGL · WN18RR | MRR: .541 -> .552 | Helped |
| Add the score retriever (ablation) | MKGL with context retriever · FB15k-237 | MRR: .365 -> .415 | Helped |
| Add the score retriever (ablation) | MKGL with context retriever · WN18RR | MRR: .512 -> .552 | Helped |
| Use text retrieval rather than learnable embeddings for context (ablation) | MKGL with context KG retrieval · FB15k-237 | MRR: .359 -> .415 | Helped |
| Use text retrieval rather than learnable embeddings for context (ablation) | MKGL with context KG retrieval · WN18RR | MRR: .492 -> .552 | Helped |
| Add context KG retrieval while using learnable context embeddings (ablation) | MKGL · FB15k-237 | MRR: .335 -> .359 | Helped |
| Add context KG retrieval while using learnable context embeddings (ablation) | MKGL · WN18RR | MRR: .466 -> .492 | Helped |
Surprising findings
- Removing the KG retriever hurts WN18RR more because many entities share identical names (e.g., 14 entities named 'call').
Limitations
- Requires more computational resources than conventional KG embedding methods.
- Performance on WN18RR is hampered by many entities with similar or identical names.
- KICGPT could not be reproduced because its source code was unavailable, so comparison relies on reported numbers.
Experimental setup
Datasets: FB15k-237 WN18RR FB15k-237-ind WN18RR-ind
Benchmarks: FB15k-237 WN18RR FB15k-237-ind WN18RR-ind
Metrics: MRR Hits@1 Hits@3 Hits@10
Baselines: TransE RotatE TuckER CompGCN DAN CoKE KG-BERT StAR KGLM FTL-LM DET KG-Llama-7b
Builds on: LoRA principal neighborhood aggregation KICGPT CoKE
Closest papers by meaning
- K-ON: Stacking Knowledge on the Head Layer of Large Language Model (AAAI 2025 · similarity 0.91)
- A Systematic Exploration of Knowledge Graph Alignment with Large Language Models in Retrieval Augmented Generation (AAAI 2025 · similarity 0.89)
- KG-FIT: Knowledge Graph Fine-Tuning Upon Open-World Knowledge (NeurIPS 2024 · similarity 0.89)
- Harnessing Large Language Models for Knowledge Graph Question Answering via Adaptive Multi-Aspect Retrieval-Augmentation (AAAI 2025 · similarity 0.88)
- LightPROF: A Lightweight Reasoning Framework for Large Language Model on Knowledge Graph (AAAI 2025 · similarity 0.88)
- KBLaM: Knowledge Base augmented Language Model (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.