Research map / Label-efficient and robust learning / Recognition and long-tail
Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase LSTM auto-encoder dimensionality (ablation) | LSTM auto-encoder · Word order prediction | accuracy | Helped |
| Increase LSTM auto-encoder dimensionality from 750 to 1000 (ablation) | LSTM auto-encoder · Word content prediction | accuracy | Hurt |
| Increase LSTM auto-encoder dimensionality beyond 300 (ablation) | LSTM auto-encoder · Sentence length prediction | accuracy | No clear effect |
| Increase CBOW dimensionality to higher dimensions (ablation) | CBOW · Word content prediction | accuracy | Hurt |
| Use 300 dimensions rather than 500 and 750 dimensions (ablation) | CBOW · Sentence length prediction | accuracy | No clear effect |
| Remove sentence embeddings from the word-order predictor (ablation) | CBOW word-order prediction · Word order prediction | accuracy: ∼3% accuracy points | Hurt |
| Remove sentence embeddings from the word-order predictor (ablation) | LSTM auto-encoder word-order prediction · Word order prediction | accuracy | Hurt |
| Randomly permute sentence word order at evaluation (ablation) | CBOW · Word order prediction | accuracy | Hurt |
| Randomly permute sentence word order at evaluation (ablation) | LSTM auto-encoder · Length, word content, and word order prediction | accuracy | No clear effect |
| Randomly permute sentence word order at evaluation (ablation) | Skip-thought · Length prediction | accuracy: 82.1% -> 68.2% | Hurt |
| Randomly permute sentence word order at evaluation (ablation) | Skip-thought · Word content prediction | accuracy: 79.7% -> 76.4% | Hurt |
| Randomly permute sentence word order at evaluation (ablation) | Skip-thought · Word order prediction | accuracy: 81.1% -> 76.5% | Hurt |
Closest papers by meaning
- No Training Required: Exploring Random Encoders for Sentence Classification (ICLR 2019 · similarity 0.91)
- A Simple but Tough-to-Beat Baseline for Sentence Embeddings (ICLR 2017 · similarity 0.90)
- Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning (ICLR 2018 · similarity 0.90)
- An efficient framework for learning sentence representations (ICLR 2018 · similarity 0.90)
- Narrowing the Gap between Supervised and Unsupervised Sentence Representation Learning with Large Language Model (AAAI 2024 · similarity 0.89)
- TNCSE: Tensor Norm Constraints for Unsupervised Contrastive Learning of Sentence Embeddings (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.