Research map / Architectures and efficiency / Attention and transformers
Jointly Learning to Label Sentences and Tokens
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| jointly supervise sentence labels and token labels | BiLSTM-ATTN · Sentence classification: CoNLL 2010, FCE, SST-neg, SST-pos | F1: 83.87 -> 87.17; 85.21 -> 86.01; 89.77 -> 91.71; 93.96 -> 96.09 | Helped |
| add token-labeling supervision (ablation) | BiLSTM-ATTN · SST-neg | F1: 89.77 -> 91.97 | Helped |
| add token-labeling supervision (ablation) | BiLSTM-ATTN · SST-pos | F1: 93.96 -> 95.82 | Helped |
| add token-labeling supervision (ablation) | BiLSTM-ATTN · CoNLL 2010 and FCE sentence classification | F1: 83.87 -> 85.90; 85.21 -> 85.41 | Helped |
| add word-level language modeling (ablation) | BiLSTM-ATTN · CoNLL 2010 and FCE sentence classification | F1: 83.87 -> 85.70; 85.21 -> 85.48 | Helped |
| add word-level language modeling (ablation) | BiLSTM-ATTN · SST-neg and SST-pos sentence classification | F1: 89.77 -> 89.71; 93.96 -> 94.02 | Mixed |
| add character-level language modeling (ablation) | BiLSTM-ATTN · CoNLL 2010, FCE, SST-neg, SST-pos sentence classification | F1: 83.87 -> 85.08; 85.21 -> 85.35; 89.77 -> 89.67; 93.96 -> 93.79 | Mixed |
| add the attention-range objective (ablation) | BiLSTM-ATTN · CoNLL 2010, FCE, SST-neg, SST-pos sentence classification | F1: 83.87 -> 83.95; 85.21 -> 85.21; 89.77 -> 89.81; 93.96 -> 93.79 | Mixed |
| jointly train sentence and token labeling with the auxiliary objectives | BiLSTM-JOINT · FCE, CoNLL14-TEST1, CoNLL14-TEST2, JFLEG error detection | F0.5: 48.48 -> 52.07; 17.86 -> 22.14; 25.88 -> 29.65; 45.74 -> 52.52 | Helped |
| use sentence-level annotation alongside limited token-level annotation | BiLSTM-JOINT · CoNLL 2010 and FCE sequence labeling | sequence-labeling performance: 20% | Helped |
Closest papers by meaning
- An efficient framework for learning sentence representations (ICLR 2018 · similarity 0.88)
- Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning (ICLR 2018 · similarity 0.88)
- Empower Sequence Labeling with Task-Aware Neural Language Model (AAAI 2018 · similarity 0.87)
- Sub-Task Decomposition Enables Learning in Sequence to Sequence Tasks (ICLR 2023 · similarity 0.87)
- Narrowing the Gap between Supervised and Unsupervised Sentence Representation Learning with Large Language Model (AAAI 2024 · similarity 0.86)
- Attention with Trained Embeddings Provably Selects Important Tokens (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.