Research map / Label-efficient and robust learning / Knowledge distillation
Distilling Knowledge from Well-Informed Soft Labels for Neural Relation Extraction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add soft type constraints (scalar weight) to C-GCN teacher | C-GCN · TACRED | F1: 66.4 -> 67.9 | Helped |
| add soft type constraints (scalar weight) to C-GCN teacher | C-GCN · SemEval | F1: 84.8 -> 85.4 | Helped |
| use trainable vector weight instead of scalar for type constraints | Teacher · TACRED | F1: 67.9 -> 68.7 | Helped |
| distill teacher soft labels into student with relation-aware teacher | Student-R · SemEval | F1: 85.9 -> 86.8 | Helped |
| add TSD loss to teacher (ablation) | Student-R · TACRED | F1: 68.8 -> 69.6 | Helped |
| add hint learning loss (ablation) | Student-R · TACRED | F1: 68.4 -> 69.6 | Helped |
| add type constraints (ablation) | Student-R · TACRED | F1: 68.0 -> 69.6 | Helped |
| add multi-aspect attention (ablation) | Student-R · TACRED | F1: 68.3 -> 69.6 | Helped |
| add knowledge distillation (vs hard labels) (ablation) | Student-R · TACRED | F1: 67.5 -> 69.6 | Helped |
| add type constraints and MAA on top of vanilla KD (ablation) | Student-R · TACRED | F1: 67.8 -> 69.6 | Helped |
| student after distillation vs its teacher (Student-R vs Teacher-R, dev excluded) (ablation) | Student-S/V · TACRED | F1: 67.9 -> 68.1 | Mixed |
| vary NA probability C for vector-weight models (ablation) | Teacher-V · TACRED | F1 | No clear effect |
Closest papers by meaning
- A Unified Multi-Task Learning Framework for Joint Extraction of Entities and Relations (AAAI 2021 · similarity 0.87)
- Aggregating Inter-Sentence Information to Enhance Relation Extraction (AAAI 2016 · similarity 0.87)
- Integrating Relation Constraints with Neural Relation Extractors (AAAI 2020 · similarity 0.87)
- Bootstrapping Distantly Supervised IE Using Joint Learning and Small Well-Structured Corpora (AAAI 2017 · similarity 0.87)
- DocKS-RAG: Optimizing Document-Level Relation Extraction through LLM-Enhanced Hybrid Prompt Tuning (ICML 2025 · similarity 0.85)
- A Positive-Unlabeled Metric Learning Framework for Document-Level Relation Extraction with Incomplete Labeling (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.