Research map / Large language models / Retrieval and knowledge
DocKS-RAG: Optimizing Document-Level Relation Extraction through LLM-Enhanced Hybrid Prompt Tuning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DocKS-RAG, which combines a GNN-built document-level knowledge graph and sentence-level retrieval into hybrid prompts for LoRA-tuned LLMs on document-level relation extraction, reporting gains on DocRED and Re-DocRED.
Problem
Document-level relation extraction struggles with modeling entity-relation structural interactions and inferring implicit cross-sentence relations; prior PLM and graph methods suffer semantic misalignment.
Approach
Combine a GNN-trained document-level knowledge graph (DocKG) and a sentence-level retrieval knowledge base (SetRAG) into hybrid textual prompts, then fine-tune an LLM with LoRA adapters on these prompts.
Key results
- On Re-DocRED test, DocKS-RAG reaches 75.49 F1 and 74.32 Ign-F1 (test Ign-F1 74.32), above GAIN, ATLOP-BERT, and DocuNet.
- On DocRED test, DocKS-RAG reports 63.59 F1 and 61.83 Ign-F1, vs. 62.14 F1 for SRLR as the strongest listed baseline.
- Ablation removing hybrid-prompt tuning drops Re-DocRED dev F1 from 75.31 to 26.87.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add DocKG (ablation) | DocKS-RAG · DocRED Test | F1: 58.08 -> 63.59 | Helped |
| add DocKG (ablation) | DocKS-RAG · Re-DocRED Test | F1: 69.03 -> 75.49 | Helped |
| add SetRAG (ablation) | DocKS-RAG · DocRED Test | F1: 54.96 -> 63.59 | Helped |
| add SetRAG (ablation) | DocKS-RAG · Re-DocRED Test | F1: 62.63 -> 75.49 | Helped |
| add Hybrid-Prompt Tuning (ablation) | DocKS-RAG · DocRED Test | F1: 20.94 -> 63.59 | Helped |
| add Hybrid-Prompt Tuning (ablation) | DocKS-RAG · Re-DocRED Test | F1: 26.32 -> 75.49 | Helped |
| increase DocKG retrieval threshold τeq from 0 to 0.7 (ablation) | DocKS-RAG · Re-DocRED Dev | F1: 36.42 -> 75.31 | Helped |
| increase DocKG retrieval threshold τeq from 0.7 to 1 (ablation) | DocKS-RAG · Re-DocRED Dev | F1: 75.31 -> 68.59 | Hurt |
| increase SetRAG Top-k from 0 to 5 (ablation) | DocKS-RAG · Re-DocRED Dev | F1: 67.78 -> 75.31 | Helped |
| increase SetRAG Top-k from 5 to 10 (ablation) | DocKS-RAG · Re-DocRED Dev | F1: 75.31 -> 68.26 | Hurt |
Surprising findings
- Removing hybrid-prompt tuning (reverting to few-shot LLM use) collapses performance to roughly 21-27 F1, far below all PLM baselines.
Limitations
- Few-shot/no-tuning LLM variant performs poorly, suggesting dependence on fine-tuning.
- Threshold hyperparameters (tau_eq, k) are tuned on dev set and sensitive to choice.
- Method description is vague about LLM size, training data size, and GNN details.
Experimental setup
Datasets: DocRED Re-DocRED
Benchmarks: DocRED Re-DocRED
Metrics: F1 Ign-F1
Baselines: HIN ATLOP-BERT SRLR GLRE LSR HeterGSAN SIRE SSAN DocuNet KD-DocRE KMGRE AutoRE
Builds on: LoRA ATLOP GAIN AutoRE
Closest papers by meaning
- Revisiting Document-Level Relation Extraction with Context-Guided Link Prediction (AAAI 2024 · similarity 0.89)
- COMM: Concentrated Margin Maximization for Robust Document-Level Relation Extraction (AAAI 2025 · similarity 0.89)
- A Hierarchical Network for Multimodal Document-Level Relation Extraction (AAAI 2024 · similarity 0.88)
- A Positive-Unlabeled Metric Learning Framework for Document-Level Relation Extraction with Incomplete Labeling (AAAI 2024 · similarity 0.88)
- Enhancing Low-Resource Relation Representations through Multi-View Decoupling (AAAI 2024 · similarity 0.87)
- An Autoregressive Text-to-Graph Framework for Joint Entity and Relation Extraction (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.