Research map / Large language models / Language, safety and interpretability
Empower Sequence Labeling with Task-Aware Neural Language Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add task-aware character-level language-model co-training with highway transformations | sequence labeling on CoNLL03 NER · CoNLL03 NER | F1: 90.76 -> 91.71 | Helped |
| add task-aware character-level language-model co-training with highway transformations | sequence labeling on WSJ POS · WSJ POS | accuracy: 97.35 -> 97.53 | Helped |
| add task-aware character-level language-model co-training with highway transformations | sequence labeling on CoNLL00 chunking · CoNLL00 chunking | F1: 94.37 -> 95.96 | Helped |
| add language-model objective while retaining highway units (ablation) | LM-LSTM-CRF on CoNLL03 NER · CoNLL03 NER | F1: 91.43±0.09 -> 91.71±0.10 | Helped |
| add highway units to mediate sequence-labeling and language-model representations (ablation) | jointly trained LM-LSTM-CRF on CoNLL03 NER · CoNLL03 NER | F1: 91.16±0.22 -> 91.71±0.10 | Helped |
| add highway units instead of directly sharing character-layer outputs (ablation) | joint language-model and sequence-labeling training on CoNLL03 NER · CoNLL03 NER | F1: 91.16±0.22 -> 91.71±0.10 | Helped |
| increase state size from 100 to 300 (ablation) | LM-LSTM-CRF on CoNLL03 NER · CoNLL03 NER | F1: 91.13±0.32 -> 91.71±0.10 | Helped |
| increase state size from 200 to 300 (ablation) | LM-LSTM-CRF on CoNLL03 NER · CoNLL03 NER | F1: 91.63±0.23 -> 91.71±0.10 | No clear effect |
Closest papers by meaning
- Transfer Learning for Sequence Labeling Using Source Model and Target Data (AAAI 2019 · similarity 0.89)
- Transfer Learning for Sequence Tagging with Hierarchical Recurrent Networks (ICLR 2017 · similarity 0.89)
- Jointly Learning to Label Sentences and Tokens (AAAI 2019 · similarity 0.87)
- Neural Models for Sequence Chunking (AAAI 2017 · similarity 0.87)
- On the Importance of Word Order Information in Cross-lingual Sequence Labeling (AAAI 2021 · similarity 0.87)
- SeqGPT: An Out-of-the-Box Large Language Model for Open Domain Sequence Understanding (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.