Research map / Label-efficient and robust learning / Recognition and long-tail
Empirical Analysis of Unlabeled Entity Problem in Named Entity Recognition
Yangming Li, lemao liu, Shuming Shi
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use negative sampling instead of treating all unlabeled spans as negatives | BERT-based span NER model · CoNLL-2003 synthetic, masking 0.6 | F1: 63.84 -> 87.65 | Improved |
| use negative sampling instead of treating all unlabeled spans as negatives | BERT-based span NER model · OntoNotes 5.0 synthetic, masking 0.6 | F1: 58.17 -> 87.53 | Improved |
| use negative sampling instead of treating all unlabeled spans as negatives | BERT-based span NER model · EC | F1: 63.56 -> 66.17 | Improved |
| use negative sampling instead of treating all unlabeled spans as negatives | BERT-based span NER model · NEWS | F1: 80.04 -> 85.39 | Improved |
| use negative sampling on fully annotated data | BERT-based span NER model · CoNLL-2003 | F1: 93.5 -> 93.42 | Hurt |
| use negative sampling on fully annotated data | BERT-based span NER model · OntoNotes 5.0 | F1: 91.3 -> 90.59 | Hurt |
| replace the BERT encoder with BiLSTM (ablation) | Our Model · EC | F1: 66.17 -> 64.68 | Hurt |
| replace the BERT encoder with BiLSTM (ablation) | Our Model · NEWS | F1: 85.39 -> 82.11 | Hurt |
| vary the negative-sampling ratio λ (ablation) | Our Model · synthetic datasets | F1 | Mixed |
| use a pretrained language model rather than LSTM to mitigate fewer annotated entities | tagging model with adjusted loss · CoNLL-2003 and OntoNotes 5.0 synthetic, masking 0.8 | erosion rate: about half | Improved |
| treat unlabeled entities as training negatives | NER models · synthetic datasets | F1: about 20% | Hurt |
Closest papers by meaning
- Robust Named Entity Recognition with Truecasing Pretraining (AAAI 2020 · similarity 0.89)
- GuideNER: Annotation Guidelines Are Better than Examples for In-Context Named Entity Recognition (AAAI 2025 · similarity 0.88)
- UniversalNER: Targeted Distillation from Large Language Models for Open Named Entity Recognition (ICLR 2024 · similarity 0.88)
- PaDeLLM-NER: Parallel Decoding in Large Language Models for Named Entity Recognition (NeurIPS 2024 · similarity 0.87)
- MCL-NER: Cross-Lingual Named Entity Recognition via Multi-View Contrastive Learning (AAAI 2024 · similarity 0.86)
- Span Graph Transformer for Document-Level Named Entity Recognition (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
