Research map / Label-efficient and robust learning / Recognition and long-tail
Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add French and German translation objectives to skip-thought next (ablation) | GenSen shared sentence encoder · Transfer benchmarks (10 tasks) | average improvement over Infersent (AllNLI): -2.56 -> 0.01 | Helped |
| add natural language inference training (ablation) | GenSen shared sentence encoder · Transfer benchmarks (10 tasks) | average improvement over Infersent (AllNLI): 0.01 -> 0.99 | Helped |
| increase encoder capacity with larger hidden units (ablation) | GenSen shared sentence encoder · Transfer benchmarks (10 tasks) | average improvement over Infersent (AllNLI): 0.99 -> 1.33 | Helped |
| add skip-thought previous objective (ablation) | GenSen shared sentence encoder · Transfer benchmarks (10 tasks) | average improvement over Infersent (AllNLI): 1.33 -> 1.44 | Helped |
| add a second encoder layer (ablation) | GenSen shared sentence encoder · Transfer benchmarks (10 tasks) | average improvement over Infersent (AllNLI): 1.44 -> 1.47 | Helped |
| add constituency parsing objective (ablation) | GenSen shared sentence encoder · SICK-R | Pearson correlation: 0.885 -> 0.888 | Helped |
| add constituency parsing objective (ablation) | GenSen shared sentence encoder · SICK-E | accuracy: 86.8 -> 87.8 | Helped |
| add parsing to the multi-task model (ablation) | GenSen shared sentence encoder · Sentence length probe | accuracy: 88.1 -> 93.7 | Helped |
| add parsing to the multi-task model (ablation) | GenSen shared sentence encoder · Word order probe | accuracy: 81.5 -> 83.1 | Helped |
| add NLI training to the multilingual skip-thought encoder (ablation) | GenSen sentence encoder · Passive syntax probe | accuracy: 97.0 -> 96.7 | Hurt |
| add NLI training to the multilingual skip-thought encoder (ablation) | GenSen sentence encoder · Tense syntax probe | accuracy: 97.0 -> 96.8 | Hurt |
| use a 2-layer encoder instead of a 1-layer large encoder (ablation) | GenSen shared sentence encoder · SST | accuracy: 84.0 -> 83.6 | Hurt |
Closest papers by meaning
- An efficient framework for learning sentence representations (ICLR 2018 · similarity 0.91)
- Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks (ICLR 2017 · similarity 0.90)
- Trans-Encoder: Unsupervised sentence-pair modelling through self- and mutual-distillations (ICLR 2022 · similarity 0.89)
- Narrowing the Gap between Supervised and Unsupervised Sentence Representation Learning with Large Language Model (AAAI 2024 · similarity 0.89)
- Jointly Learning to Label Sentences and Tokens (AAAI 2019 · similarity 0.88)
- An Information-theoretic Multi-task Representation Learning Framework for Natural Language Understanding (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.