Research map / Large language models / Retrieval and knowledge
LLM-DR: A Novel LLM-Aided Diffusion Model for Rule Generation on Temporal Knowledge Graphs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LLM-DR, a rule-based TKG extrapolation method that uses a classifier-free-guided diffusion model to generate new logic rules, LLM-based relation constraints and plausibility filtering, and BERT-based scoring.
Problem
Rule-based temporal knowledge graph extrapolation suffers from a vast rule search space, making it hard to find high-quality logical rules, and LLM-generated rules are unstable and hallucination-prone.
Approach
Generate new temporal logic rule bodies with a conditional diffusion model (classifier-free guidance on the rule head), constrain candidate relations with LLM selection, and evaluate rules coarse-to-fine: LLM chain-of-thought plausibility filtering followed by BERT cosine-similarity scoring.
Key results
- LLM-DR reaches 50.5 MRR on ICEWS14 and 38.2 MRR on ICEWS18, outperforming LogCL (48.9 and 35.7) and other listed baselines.
- On GDELT LLM-DR achieves 30.7 MRR versus 23.8 for LogCL, the strongest baseline listed.
- Ablation: removing diffusion-generated rules drops ICEWS14 MRR from 50.5 to 43.8.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use LLM-DR instead of LogCL | TKG extrapolation · ICEWS14 | MRR: 48.9 -> 50.5 | Helped |
| use LLM-DR instead of LogCL | TKG extrapolation · ICEWS18 | MRR: 35.7 -> 38.2 | Helped |
| use LLM-DR instead of LogCL | TKG extrapolation · ICEWS05-15 | MRR: 57.0 -> 58.9 | Helped |
| use LLM-DR instead of LogCL | TKG extrapolation · GDELT | MRR: 23.8 -> 30.7 | Helped |
| add generation constraints (ablation) | LLM-DR · ICEWS14 | MRR: 49.4 -> 50.5 | Helped |
| add generation constraints (ablation) | LLM-DR · ICEWS18 | MRR: 36.7 -> 38.2 | Helped |
| add diffusion-generated rules (ablation) | LLM-DR · ICEWS14 | MRR: 43.8 -> 50.5 | Helped |
| add diffusion-generated rules (ablation) | LLM-DR · ICEWS18 | MRR: 31.2 -> 38.2 | Helped |
| use diffusion rather than LLM for rule generation (ablation) | LLM-DR · ICEWS14 | MRR: 46.6 -> 50.5 | Helped |
| use diffusion rather than LLM for rule generation (ablation) | LLM-DR · ICEWS18 | MRR: 35.7 -> 38.2 | Helped |
| add coarse-grained rule evaluation (ablation) | LLM-DR · ICEWS14 | MRR: 48.8 -> 50.5 | Helped |
| add coarse-grained rule evaluation (ablation) | LLM-DR · ICEWS18 | MRR: 36.6 -> 38.2 | Helped |
Surprising findings
- Replacing BERT with an LLM in the fine-grained scoring module substantially degrades performance (ICEWS14 MRR 50.5 to 46.2), suggesting LLMs are poorer at fine-grained scoring.
- Diffusion-based generation outperforms LLM-based rule generation (ICEWS14 MRR 46.6 for the LLM variant vs 50.5).
Limitations
- Relies on GPT-3.5-turbo for constraints and coarse evaluation, which adds API dependence and cost.
- Rule-based methods remain weaker than embedding methods on Hits@10 per the authors' own discussion.
Experimental setup
Datasets: ICEWS14 ICEWS18 ICEWS05-15 GDELT
Benchmarks: ICEWS14 ICEWS18 ICEWS05-15 GDELT
Metrics: MRR Hits@1 Hits@3 Hits@10
Baselines: CyGNet TITer RE-GCN xERTE TiRGN HiSMatch RPC LogCL TLogic TR-Rules Llama-2-7b-CoH Vicuna-7b-CoH
Builds on: TLogic classifier-free guidance denoising diffusion probabilistic models Chain-of-thought prompting
Closest papers by meaning
- Large Language Models-guided Dynamic Adaptation for Temporal Knowledge Graph Reasoning (NeurIPS 2024 · similarity 0.90)
- RuAG: Learned-rule-augmented Generation for Large Language Models (ICLR 2025 · similarity 0.88)
- Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph (ICLR 2024 · similarity 0.87)
- Latent Logic Tree Extraction for Event Sequence Explanation from LLMs (ICML 2024 · similarity 0.86)
- DARG: Dynamic Evaluation of Large Language Models via Adaptive Reasoning Graph (NeurIPS 2024 · similarity 0.86)
- GIVE: Structured Reasoning of Large Language Models with Knowledge Graph Inspired Veracity Extrapolation (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.