Research map / Continual learning / Class-incremental
Prototypical Replay with Old-class Focusing Knowledge Distillation for Incremental Named Entity Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes POF, which replays stored per-entity-type feature prototypes (with Gaussian noise) and applies an old-class-focused feature distillation loss to reduce catastrophic forgetting in incremental NER.
Problem
Catastrophic forgetting in incremental named entity recognition (INER): distillation-only methods leave the new model confused about old entity types, causing false positives, while raw-text replay needs large storage.
Approach
Store compact per-entity-type feature statistics (direction mean/std and norm mean/std) and replay sampled pseudo-features through the classifier at frequencies set by entity token counts, combined with a distillation loss applied only to old-class token regions (defined by ground truth/pseudo-labels) so new classes remain plastic.
Key results
- On Few-NERD FG-6-PG-6, POF reaches 48.41 final Macro-F1 (AT), a 9.73-point gain over the best prior baseline RDP, and 49.25 average (7.12-point gain).
- On I2B2 FG-12-PG-1, POF attains AT 72.78 and average 75.70, versus IS3 at 69.14 and 72.74.
- Ablation on FG-12-PG-1 shows removing prototypical replay drops I2B2 AT by 17.77 points; removing OFKD drops I2B2 AT by 6.03 points.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add prototypical replay (PR) (ablation) | POF · I2B2 FG-12-PG-1 | Macro-F1 (AT): +17.77% | Helped |
| Add prototypical replay (PR) (ablation) | POF · OntoNotes5 FG-12-PG-1 | Macro-F1 (AT): +6.21% | Helped |
| Add old-class focusing knowledge distillation (OFKD) (ablation) | POF · I2B2 FG-12-PG-1 | Macro-F1 (AT): 6.03% AT | Helped |
| Add old-class focusing knowledge distillation (OFKD) (ablation) | POF · OntoNotes5 FG-12-PG-1 | Macro-F1 (AT): 2.89% AT | Helped |
| Use POF instead of the strongest reported baseline | BERT incremental NER · Few-NERD FG-6-PG-6 | Macro-F1 (AT): ⇑9.73% | Helped |
| Use POF instead of the strongest reported baseline | BERT incremental NER · I2B2 FG-8-PG-1 | Macro-F1 (AT): ⇑7.20% | Helped |
| Use POF instead of the strongest reported baseline | BERT incremental NER · OntoNotes5 FG-12-PG-1 | Macro-F1 (AT): ⇑6.50% | Helped |
Surprising findings
- The paper reports that simply replaying per-class mean features (as in IS3) lacks the diversity needed, so storing standard deviations and norm statistics matters for sampling.
Limitations
- Relies on pseudo-labels from the old model, so errors can propagate across steps.
- Prototypes are Gaussian approximations of feature distributions, which may not capture multimodal entity features.
Experimental setup
Datasets: Few-NERD I2B2 OntoNotes5
Benchmarks: Few-NERD I2B2 OntoNotes5
Metrics: Macro-F1 on last task (AT) average Macro-F1 across tasks (A-bar)
Baselines: PODNet LUCIR ST ExtendNER CFNER DLD RDP IS3
Builds on: ExtendNER CFNER RDP IS3 LUCIR
Closest papers by meaning
- Unify Named Entity Recognition Scenarios via Contrastive Real-Time Updating Prototype (AAAI 2024 · similarity 0.87)
- UniversalNER: Targeted Distillation from Large Language Models for Open Named Entity Recognition (ICLR 2024 · similarity 0.85)
- ConsistNER: Towards Instructive NER Demonstrations for LLMs with the Consistency of Ontology and Context (AAAI 2024 · similarity 0.85)
- GuideNER: Annotation Guidelines Are Better than Examples for In-Context Named Entity Recognition (AAAI 2025 · similarity 0.85)
- Towards Continual Knowledge Graph Embedding via Incremental Distillation (AAAI 2024 · similarity 0.84)
- Adaptive Prototype Replay for Class Incremental Semantic Segmentation (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.