Research map / Multi-agent systems and games / LLM agents
A Multi-Agent LLM Framework for Multi-Domain Low-Resource In-Context NER via Knowledge Retrieval, Disambiguation and Reflective Analysis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes KDR-Agent, a multi-agent prompting framework for multi-domain low-resource in-context NER that combines type definitions, static entity-level contrastive demonstrations, Wikipedia knowledge retrieval, disambiguation, and reflective self-correction.
Problem
In-context NER in multi-domain low-resource settings suffers from reliance on retrieving annotated demonstrations, insufficient LLM domain knowledge for unseen domains, and failure to inject external knowledge or resolve entity ambiguity.
Approach
Replaces retrieved demonstrations with a static set of span-level positive-negative contrastive examples plus natural-language type definitions, and adds a central planner that dispatches Wikipedia retrieval and disambiguation agents followed by a reflection agent that corrects predictions via error categories.
Key results
- KDR-Agent with GPT-4o reaches 82.47 F1 on BC5CDR and 80.78 on WNUT-17, outperforming the strongest few-shot baseline CodeIE (77.61 and 69.91).
- With Qwen-2.5-72B, KDR-Agent achieves 81.45 F1 on BC5CDR versus 76.10 for CodeIE.
- Ablation on GPT-4o shows removing the reflection stage drops NCBI F1 from 79.41 to 75.91 and Twitter NER-7 from 60.87 to 57.81.
Surprising findings
- Removing the entity-level negative contrastive examples causes only modest declines (e.g., NCBI 79.41 to 78.36), suggesting limited individual contribution.
Limitations
- Relies on Wikipedia retrieval, so benefit depends on coverage of domain concepts.
- Performance drops noticeably with smaller backbones, especially on biomedical and social media data.
- Evaluation of efficiency/latency is claimed but not measured.
Experimental setup
Datasets: BC5CDR NCBI MIT Movie MIT Restaurant CoNLL-2003 OntoNotes 5.0 Twitter Broad Twitter NER-7 WikiANN WNUT-17
Benchmarks: BC5CDR NCBI MIT Movie MIT Restaurant CoNLL-2003 OntoNotes 5.0 Twitter Broad Twitter NER-7 WikiANN WNUT-17
Metrics: F1 span error rate type error rate spurious detection rate omission rate
Baselines: ChatIE Self-Improving CMAS GPT-NER CodeIE
Builds on: GPT-NER CodeIE ChatIE CMAS
Closest papers by meaning
- GuideNER: Annotation Guidelines Are Better than Examples for In-Context Named Entity Recognition (AAAI 2025 · similarity 0.88)
- ConsistNER: Towards Instructive NER Demonstrations for LLMs with the Consistency of Ontology and Context (AAAI 2024 · similarity 0.87)
- UniversalNER: Targeted Distillation from Large Language Models for Open Named Entity Recognition (ICLR 2024 · similarity 0.86)
- Three Heads Are Better than One: Improving Cross-Domain NER with Progressive Decomposed Network (AAAI 2024 · similarity 0.86)
- MCL-NER: Cross-Lingual Named Entity Recognition via Multi-View Contrastive Learning (AAAI 2024 · similarity 0.86)
- KARMA: Leveraging Multi-Agent LLMs for Automated Knowledge Graph Enrichment (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.