Research map / Large language models / Language, safety and interpretability

PaDeLLM-NER: Parallel Decoding in Large Language Models for Named Entity Recognition

NeurIPS 2024 · Large language models › Language, safety and interpretability · approach: Parallel and early-exit decoding · 5 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes PaDeLLM-NER, which reformulates NER instruction tuning so each LLM sequence generates one label-mention pair, enabling parallel decoding that is 1.76-10.22x faster than autoregressive generation with comparable F-scores.

Problem

LLM-based NER generates all labels and mentions autoregressively, producing long output sequences and high inference latency.

Approach

Split each NER training example into per-label sequences that predict a mention count and then the n-th mention of that label; at inference, predict counts first and then decode all label-mention pairs in parallel (multi-GPU or batched), removing duplicate mentions by token-probability.

Key results

• Inference is 1.76x to 10.22x faster than autoregressive baselines; PaDeLLM-Multi averages 223.57 ms vs 1,015.12 ms for AutoReg-Struct.
• Average generated sequence length is about 13% of autoregressive output (4.86 vs 35.54-36.48 tokens).
• Supervised micro F1 averages 85.07 on English (vs 84.25 AutoReg-Struct) and 84.66 on Chinese (vs 82.02 AutoReg-Struct); zero-shot average 61.68, second to GNER-LLaMA-7B (66.05).

Measured effects reported in the paper

Surprising findings

• Batch decoding on a single GPU was slower than multi-GPU parallel decoding (378.40 ms vs 223.57 ms).
• Supervised encoder-based methods outperform the LLM-based methods on most supervised benchmarks.

Limitations

• Training examples multiply from one to m*n per input, increasing training cost.
• LLMs struggle to count mentions accurately (count mismatch was 56.8% of sampled errors on ACE05).
• The de-duplication step can remove valid mentions that legitimately appear under multiple labels, and reformulation loses location information.

Experimental setup

Datasets: Pile-NER CrossNER MIT CoNLL2003 ACE2005 GENIA Resume Weibo MSRA OntoNotes 4.0 Youku Ecommerce

Benchmarks: CoNLL2003 ACE2005 GENIA Weibo MSRA OntoNotes 4.0 Resume Youku Ecommerce CrossNER MIT

Metrics: micro F-score latency (ms) average generated sequence length

Baselines: AutoReg-Aug AutoReg-Struct GoLLIE-7B UniNER-7B GLiNER-L GNER-LLaMA-7B BINDER DeepStruct W2NER SSCNN NEZHA-BC

Builds on: UniNER GNER instruction tuning speculative sampling

Closest papers by meaning

• UniversalNER: Targeted Distillation from Large Language Models for Open Named Entity Recognition (ICLR 2024 · similarity 0.88)
• Accelerating LLM Inference with Lossless Speculative Decoding Algorithms for Heterogeneous Vocabularies (ICML 2025 · similarity 0.87)
• Small Language Model Makes an Effective Long Text Extractor (AAAI 2025 · similarity 0.87)
• ConsistNER: Towards Instructive NER Demonstrations for LLMs with the Consistency of Ontology and Context (AAAI 2024 · similarity 0.86)
• MCL-NER: Cross-Lingual Named Entity Recognition via Multi-View Contrastive Learning (AAAI 2024 · similarity 0.86)
• RAPID: Long-Context Inference with Retrieval-Augmented Speculative Decoding (ICML 2025 · similarity 0.86)