Research map / Label-efficient and robust learning / Recognition and long-tail
A Positive-Unlabeled Metric Learning Framework for Document-Level Relation Extraction with Incomplete Labeling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes P^3M, a positive-unlabeled metric learning framework for document-level relation extraction with incomplete labels, using dropout-based positive augmentation and positive-none-class mixup.
Problem
Document-level relation extraction suffers from incomplete labeling (many false negatives), causing distribution bias in positive samples and poor generalization when training with standard supervised losses.
Approach
Formulates document-level RE as proxy-based metric learning (pull entity pairs toward their relation embedding, push from the none-class embedding), adapts it to a PU risk estimator with prior shift and non-negative correction, augments positives via dropout, and mixes positive embeddings with the none-class embedding as pseudo-negatives.</1-2 sentences>
Key results
- On DocRED with BERT-Base, P3M reaches 61.03 F1 vs 56.14 for SSR-PU under incomplete labeling; with RoBERTa-Large it reaches 64.34 F1.
- On DocRED_ext (extreme incomplete labeling), P3M improves F1 by 9.78 (BERT-Base) and 10.12 (RoBERTa-Large) over SSR-PU.
- On ChemDisGene All relationships test, P3M reaches 53.62 F1, a 5.06 gain over SSR-PU (48.56).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use P³M instead of SSR-PU | ATLOP+BERT_Base · DocRED incomplete-label training, Re-DocRED revised test | F1: +4.89 | Helped |
| Use P³M instead of SSR-PU | ATLOP+RoBERTa_Large · DocRED incomplete-label training, Re-DocRED revised test | F1: +4.84 | Helped |
| Use P³M instead of SSR-PU | ATLOP+BERT_Base · DocRED_ext extreme incomplete-label training, Re-DocRED revised test | F1: +9.78 | Helped |
| Use P³M instead of SSR-PU | ATLOP+RoBERTa_Large · DocRED_ext extreme incomplete-label training, Re-DocRED revised test | F1: +10.12 | Helped |
| Use P³M instead of SSR-PU | PubmedBert · ChemDisGene All relationships test | F1: +5.06 | Helped |
| Augment all samples rather than positive samples only (ablation) | P²M · DocRED_ext extreme incomplete-label training | F1 | Hurt |
| Augment all samples rather than positive samples only (ablation) | PM · ChemDisGene All relationships test | F1: 52.02 -> 51.29 | Hurt |
| Add dropout augmentation of positive samples (ablation) | PM · ChemDisGene All relationships test | F1: 52.02 -> 52.19 | Helped |
| Add positive-none-class mixup (ablation) | P²M · ChemDisGene All relationships test | F1: 52.19 -> 53.62 | Helped |
| Use none-class relation embedding rather than unlabeled samples for mixup (ablation) | P³M(ori) · DocRED incomplete-label training | F1 | Helped |
| Use none-class relation embedding rather than unlabeled samples for mixup (ablation) | P³M(ori) · ChemDisGene All relationships test | F1: 53.58 -> 53.62 | No clear effect |
| Use P³M instead of SSR-PU | BERT_Base · Re-DocRED revised test, fully supervised | F1: 74.33 -> 75.60 | Helped |
Surprising findings
- Augmenting all samples with dropout (P2M(all)) hurts performance under extreme incomplete labeling, while augmenting only positives helps.
- Replacing unlabeled-sample mixup with none-class embedding mixup gives a consistent gain over P3M(ori).
Limitations
- Precision drops slightly relative to SSR-PU on DocRED, attributed to expanded positive distribution.
- Relies on estimated class priors; performance depends on prior estimation, though reported as robust.
Experimental setup
Datasets: DocRED DocRED_ext Re-DocRED ChemDisGene
Benchmarks: DocRED Re-DocRED ChemDisGene
Metrics: F1 Ign F1 precision recall
Baselines: BiLSTM GAIN DocuNET ATLOP SSR-PU BRAN PubmedBert KD-DocRE
Builds on: ATLOP SSR-PU SoftTriplet loss non-negative PU risk estimation mixup
Closest papers by meaning
- COMM: Concentrated Margin Maximization for Robust Document-Level Relation Extraction (AAAI 2025 · similarity 0.88)
- DocKS-RAG: Optimizing Document-Level Relation Extraction through LLM-Enhanced Hybrid Prompt Tuning (ICML 2025 · similarity 0.88)
- A Hierarchical Network for Multimodal Document-Level Relation Extraction (AAAI 2024 · similarity 0.87)
- An Autoregressive Text-to-Graph Framework for Joint Entity and Relation Extraction (AAAI 2024 · similarity 0.86)
- Revisiting Document-Level Relation Extraction with Context-Guided Link Prediction (AAAI 2024 · similarity 0.86)
- Synergistic Anchored Contrastive Pre-training for Few-Shot Relation Extraction (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.