Research map / Label-efficient and robust learning / Self-supervised and contrastive
An efficient framework for learning sentence representations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace context-sentence word generation with contrastive classification over candidate sentence embeddings | Quick Thoughts sentence encoders · downstream NLP tasks | training time: 6.5 hrs vs 31 hrs; 4.8x | Helped |
| train on BookCorpus + UMBC instead of BookCorpus alone | MC-QT · downstream NLP tasks | task performance | Helped |
| add a fixed pretrained-word-vector channel alongside a trainable-from-scratch channel (ablation) | QT sentence encoders · MR | accuracy: 81.3 -> 82.4 | Helped |
| combine the uni-QT and bi-QT representations (ablation) | QT sentence encoders · CR (BookCorpus) | accuracy: 82.8 / 83.5 -> 84.4 | Helped |
| use a bidirectional rather than unidirectional encoder (ablation) | QT sentence encoders · CR (BookCorpus) | accuracy: 82.8 -> 83.5 | Helped |
| increase MC-QT embedding size from 1600 to 4800 (ablation) | MC-QT · classification benchmarks | mean accuracy: 84.57 -> 86.43 | Helped |
| increase training corpus from BookCorpus to BookCorpus + UMBC (ablation) | MC-QT · TREC | accuracy: 92.8 -> 92.4 | No clear effect |
| use QT (RNN) instead of Skip-Thoughts | sentence representations · STS14 Overall | Pearson score: 0.39 -> 0.49 | Helped |
| use QT (BoW) instead of FastSent | sentence representations · STS14 Overall | Pearson score: 0.64 -> 0.65 | Helped |
| use MC-QT instead of Combine-skip | sentence representations · COCO image annotation retrieval | median rank: 3 -> 2 | Helped |
| use MC-QT instead of InferSent | sentence representations · SICK | Pearson r: 0.884 -> 0.874 | Hurt |
| replace multiclass candidate classification with a binary plausible/implausible-window objective (ablation) | QT training objective · context prediction | prediction performance | Hurt |
Closest papers by meaning
- Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning (ICLR 2018 · similarity 0.91)
- No Training Required: Exploring Random Encoders for Sentence Classification (ICLR 2019 · similarity 0.90)
- Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks (ICLR 2017 · similarity 0.90)
- Narrowing the Gap between Supervised and Unsupervised Sentence Representation Learning with Large Language Model (AAAI 2024 · similarity 0.89)
- TNCSE: Tensor Norm Constraints for Unsupervised Contrastive Learning of Sentence Embeddings (AAAI 2025 · similarity 0.86)
- DenoSent: A Denoising Objective for Self-Supervised Sentence Representation Learning (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.