Research map / Large language models / Language, safety and interpretability
Inducing Relational Knowledge from BERT
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use automatically filtered templates with BERT^max (K=100) instead of Skip-gram Trans | Relation induction · Google | F1: 73.6 -> 77.0 | Helped |
| Use automatically filtered templates with BERT^max (K=100) instead of Skip-gram Trans | Relation induction · DiffVec | F1: 57.2 -> 52.8 | Hurt |
| Use automatically filtered templates with BERT^max (K=100) instead of Skip-gram Trans | Relation induction · BATS | F1: 66.6 -> 49.5 | Hurt |
| Select 100 rather than 50 templates (ablation) | BERT^max · Google | F1: 75.0 -> 77.0 | Helped |
| Select 100 rather than 50 templates (ablation) | BERT^max · DiffVec | F1: 49.6 -> 52.8 | Helped |
| Select 100 rather than 50 templates (ablation) | BERT^max · BATS | F1: 44.5 -> 49.5 | Helped |
| Filter templates with the proposed scoring rather than using 1000 unfiltered templates (ablation) | BERT^max · Google | F1: 65.8 -> 77.0 | Helped |
| Filter templates with the proposed scoring rather than using 1000 unfiltered templates (ablation) | BERT^max · DiffVec | F1: 45.7 -> 52.8 | Helped |
| Filter templates with the proposed scoring rather than using 1000 unfiltered templates (ablation) | BERT^max · BATS | F1: 44.7 -> 49.5 | Helped |
| Use thresholded average aggregation instead of BERT^max aggregation (ablation) | BERT relation classifier · Google | F1: 77.0 -> 70.6 | Hurt |
| Use thresholded average aggregation instead of BERT^max aggregation (ablation) | BERT relation classifier · DiffVec | F1: 52.8 -> 54.4 | Helped |
| Use thresholded average aggregation instead of BERT^max aggregation (ablation) | BERT relation classifier · BATS | F1: 49.5 -> 50.8 | Helped |
Closest papers by meaning
- Inductive Relation Prediction by BERT (AAAI 2022 · similarity 0.89)
- K-BERT: Enabling Language Representation with Knowledge Graph (AAAI 2020 · similarity 0.88)
- Interpreting Knowledge Graph Relation Representation from Word Embeddings (ICLR 2021 · similarity 0.88)
- Linearity of Relation Decoding in Transformer Language Models (ICLR 2024 · similarity 0.86)
- An Autoregressive Text-to-Graph Framework for Joint Entity and Relation Extraction (AAAI 2024 · similarity 0.86)
- Physics of Language Models: Part 3.1, Knowledge Storage and Extraction (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.