Research map / Label-efficient and robust learning / Recognition and long-tail
PMRC: Prompt-Based Machine Reading Comprehension for Few-Shot Named Entity Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Reformulates few-shot NER as prompt-augmented machine reading comprehension, inserting per-entity-type start/end marker tokens into templates and using dot-product matching to extract all entity spans in one pass.
Problem
Few-shot NER with prompting typically requires hand-designed templates and enumerating all candidate spans, giving high computational cost in training and inference, and poor cross-domain transfer when label sets change.
Approach
Insert [ENT_START]/[ENT_END] boundary markers for each entity type (filled with label words or example-based text) into the input, then predict start/end positions by dot-product between token representations and marker representations, with no extra classifier parameters, so the same model transfers to new label sets by swapping templates and extracts all entity types in one round.
Key results
- In the cross-domain 10-shot ATIS setting without source data, PMRC reaches 92.0 F1 versus LightNER's 76.3, a +15.7 gain.
- Averaged over cross-domain few-shot settings without source data, PMRC improves about +5.2% over prior SOTA as reported.
- Inference on CoNLL03 is 40.56x faster than TemplateNER and 2.44x faster than LightNER.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PMRC instead of LightNER | few-shot NER · ATIS 10-shot, no source data | F1: 76.3 -> 92.0 | Helped |
| Use PMRC instead of LightNER | few-shot NER · MIT Movie 10-shot, no source data | F1: 41.7 -> 58.2 | Helped |
| Use PMRC instead of LightNER | few-shot NER · MIT Restaurant 500-shot, no source data | F1: 80.2 -> 77.8 | Hurt |
| Add CoNLL03 source-domain training | PMRC · cross-domain few-shot NER, average | F1: +2% | Helped |
| Add an MLP layer to distinguish entity start and end positions (ablation) | PMRC · MIT Movie 10-shot, no source data | F1: 56.0 -> 58.2 | Helped |
| Add label words to prompt templates (ablation) | PMRC · MIT Movie 10-shot, no source data | F1: 52.7 -> 58.2 | Helped |
| Use Focal Loss instead of cross-entropy loss (ablation) | PMRC · MIT Movie 10-shot, no source data | F1: 57.7 -> 58.2 | Helped |
| Add an MLP layer to distinguish entity start and end positions (ablation) | PMRC · MIT Movie 10-shot, CoNLL03 source | F1: 62.7 -> 65.6 | Helped |
| Add label words to prompt templates (ablation) | PMRC · MIT Movie 10-shot, CoNLL03 source | F1: 60.0 -> 65.6 | Helped |
| Use Focal Loss instead of cross-entropy loss (ablation) | PMRC · MIT Movie 10-shot, CoNLL03 source | F1: 61.2 -> 65.6 | Helped |
| Use an example-based template instead of a label-word template (ablation) | PMRC · MIT Movie, 5-shot | F1: 41.5; 44.1 | Mixed |
| Use PMRC instead of TemplateNER | NER inference · CoNLL03 | speedup: 40.56× faster | Helped |
Surprising findings
- Removing label words from the template causes the largest ablation drop (5.6 points at 10-shot with source data on MIT Movie).
- Example-based templates perform comparably to label-word templates, and random-word templates are only slightly worse, suggesting PMRC is robust to template choice.
Limitations
- Performs worse on the MISC entity type in the in-domain few-shot setting than LightNER.
- Example-based templates grow excessively long when there are many entity classes, causing input length overflow.
- Standard supervised CoNLL03 F1 (91.86) is slightly below the best prior results and only competitive with LightNER (92.93).
Experimental setup
Datasets: CoNLL03 MIT Movie MIT Restaurant ATIS
Benchmarks: CoNLL03 MIT Movie MIT Restaurant ATIS
Metrics: F1 precision recall inference speedup
Baselines: BERTTagger TemplateNER LightNER BERT-MRC Neigh.Tag Example-based NER MP-NSP QaNER
Builds on: NER-MQMRC TemplateNER LightNER COPNER Label Semantics for Few Shot NER
Closest papers by meaning
- Multi-Grained Query-Guided Set Prediction Network for Grounded Multimodal Named Entity Recognition (AAAI 2025 · similarity 0.85)
- UniversalNER: Targeted Distillation from Large Language Models for Open Named Entity Recognition (ICLR 2024 · similarity 0.85)
- ConsistNER: Towards Instructive NER Demonstrations for LLMs with the Consistency of Ontology and Context (AAAI 2024 · similarity 0.85)
- PaDeLLM-NER: Parallel Decoding in Large Language Models for Named Entity Recognition (NeurIPS 2024 · similarity 0.84)
- GuideNER: Annotation Guidelines Are Better than Examples for In-Context Named Entity Recognition (AAAI 2025 · similarity 0.84)
- Prompt Sketching for Large Language Models (ICML 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.