Research map / Label-efficient and robust learning / Recognition and long-tail
Seed-Guided Fine-Grained Entity Typing in Science and Engineering Domains
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SEType, a seed-guided fine-grained entity typing framework for software and security domains that enriches seed entities via PLM embeddings and trains an NLI-style entailment model on pseudo-labeled data to classify seen and unseen types.
Problem
Fine-grained entity typing in specialized science and engineering domains with only type names and a few seed entities per type, with no context-annotated data, covering both seen and unseen types.
Approach
Enrich each type's seed entities with mutually exclusive expansion using contextualized BERTOverflow embeddings (content+context), match enriched entities to unlabeled text to create pseudo-labels, and train a cross-encoder entailment model with a contrastive loss so unseen types can be scored via their names as hypotheses.
Key results
- SEType reaches 66.15 closed-set Micro-F1 on StackOverflow versus 50.79 for GPT-3.5 Turbo and 61.37 for CGExpan+Entailment.
- On NVD, SEType achieves 75.93 closed-set Micro-F1, the best Micro-F1 among compared methods, and 61.69 open-set Micro-F1.
- On Metasploit, SEType achieves 74.77 closed-set Micro-F1 versus 71.70 for SetExpan+Entailment.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SEType instead of the strongest listed baseline | Seed-guided entity typing · StackOverflow closed-set | Micro-F1: 48.99 -> 66.15 | Helped |
| Use SEType instead of the strongest listed baseline | Seed-guided entity typing · StackOverflow open-set | Micro-F1: 51.86 -> 60.05 | Helped |
| Use SEType instead of the strongest listed baseline | Seed-guided entity typing · GitHub closed-set | Micro-F1: 45.97 -> 52.30 | Helped |
| Use SEType instead of the strongest listed baseline | Seed-guided entity typing · GitHub open-set | Micro-F1: 49.85 -> 52.45 | Helped |
| Use SEType instead of the strongest listed baseline | Seed-guided entity typing · NVD closed-set | Micro-F1: 66.86 -> 75.93 | Helped |
| Use SEType instead of the strongest listed baseline | Seed-guided entity typing · NVD open-set | Micro-F1: 50.45 -> 61.69 | Helped |
| Use SEType instead of the strongest listed baseline | Seed-guided entity typing · Metasploit closed-set | Micro-F1: 71.70 -> 74.77 | Helped |
| Use SEType instead of the strongest listed baseline | Seed-guided entity typing · Metasploit open-set | Micro-F1: 50.19 -> 62.50 | Helped |
| Add one sentence of context on each side of the mention sentence (c=1 rather than c=0) (ablation) | SEType · StackOverflow and GitHub | F1 | Helped |
| Increase enriched entities per type from 20 to 50 (ablation) | SEType · StackOverflowNER and Cybersecurity | F1 | Helped |
| Increase enriched entities per type from 100 to 200 (ablation) | SEType · StackOverflowNER and Cybersecurity | F1 | Hurt |
| Use the Taxonomic template instead of the Contextual template (ablation) | SEType · StackOverflow, GitHub, NVD, and Metasploit | Micro-F1 | No clear effect |
Surprising findings
- Plain BERTOverflow few-shot typing without entity enrichment is often lower than zero-shot RoBERTa-large-MNLI and BART-large-MNLI.
- SEType does not beat SetExpan+Entailment on NVD and Metasploit Macro-F1 scores, where it is second-best.
Limitations
- Performance on Macro-F1 for NVD and Metasploit trails SetExpan+Entailment and GPT-3.5 Turbo in some settings.
- Enriched entity count and context window require tuning; performance drops when too many entities are added.
- Evaluated only on software and security domains with 10-15 fine-grained types.
Experimental setup
Datasets: StackOverflowNER Cybersecurity (NVD and Metasploit) Stack Exchange data dump
Benchmarks: StackOverflowNER (StackOverflow, GitHub) Cybersecurity (NVD, Metasploit)
Metrics: Micro-F1 Macro-F1
Baselines: RoBERTa-large-MNLI BART-large-MNLI BERTOverflow SetExpan+Entailment CGExpan+Entailment GPT-3.5 Turbo
Builds on: SetExpan CGExpan BERTOverflow AutoPhrase RoBERTa-large-MNLI
Closest papers by meaning
- UniversalNER: Targeted Distillation from Large Language Models for Open Named Entity Recognition (ICLR 2024 · similarity 0.84)
- KGGen: Extracting Knowledge Graphs from Plain Text with Language Models (NeurIPS 2025 · similarity 0.84)
- Select and Augment: Enhanced Dense Retrieval Knowledge Graph Augmentation (Abstract Reprint) (AAAI 2024 · similarity 0.83)
- MESED: A Multi-Modal Entity Set Expansion Dataset with Fine-Grained Semantic Classes and Hard Negative Entities (AAAI 2024 · similarity 0.83)
- HLMEA: Unsupervised Entity Alignment Based on Hybrid Language Models (AAAI 2025 · similarity 0.83)
- GuideNER: Annotation Guidelines Are Better than Examples for In-Context Named Entity Recognition (AAAI 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.