Research map / Label-efficient and robust learning / Recognition and long-tail
Instance Smoothed Contrastive Learning for Unsupervised Sentence Embedding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add instance smoothing with retrieved similar embeddings | BERT-base unsup-SimCSE · STS12-16, STS-B, SICK-R (Avg.) | Spearman's correlation: 76.25 -> 78.30 | Helped |
| Add instance smoothing with retrieved similar embeddings | BERT-large unsup-SimCSE · STS12-16, STS-B, SICK-R (Avg.) | Spearman's correlation: 78.41 -> 79.47 | Helped |
| Add instance smoothing with retrieved similar embeddings | RoBERTa-base unsup-SimCSE · STS12-16, STS-B, SICK-R (Avg.) | Spearman's correlation: 76.57 -> 77.73 | Helped |
| Add instance smoothing with retrieved similar embeddings | RoBERTa-large unsup-SimCSE · STS12-16, STS-B, SICK-R (Avg.) | Spearman's correlation: 78.90 -> 79.42 | Helped |
| Add the smoothing loss to DCLR | DCLR-RoBERTa-large (reproduced) · STS12-16, STS-B, SICK-R (Avg.) | Spearman's correlation: 78.95 -> 79.49 | Helped |
| Add the smoothing method to CARDS | CARDS-RoBERTa-large · STS12-16, STS-B, SICK-R (Avg.) | Spearman's correlation: 80.46 -> 80.77 | Helped |
| Use kNN instead of k-means retrieval (ablation) | IS-CSE-BERT-base · STS-B development | Spearman's correlation: 83.74 -> 84.18 | Helped |
| Use both kNN and k-means groups (ablation) | IS-CSE-BERT-base · STS-B development | Spearman's correlation: 84.18 -> 84.14 | No clear effect |
| Increase buffer size from 256 to 1024 (ablation) | IS-CSE-BERT-base · STS-B development | Spearman's correlation: 82.75 -> 84.18 | Helped |
| Increase buffer size beyond 1024 to 1536 (ablation) | IS-CSE-BERT-base · STS-B development | Spearman's correlation: 84.18 -> 83.02 | Hurt |
| Increase the neighbor count from 8 to 16 (ablation) | IS-CSE-BERT-base · STS-B development | Spearman's correlation: 83.18 -> 84.18 | Helped |
| Increase the neighbor count from 16 to 24 (ablation) | IS-CSE-BERT-base · STS-B development | Spearman's correlation: 84.18 -> 82.63 | Hurt |
Closest papers by meaning
- A Simple but Tough-to-Beat Baseline for Sentence Embeddings (ICLR 2017 · similarity 0.90)
- TNCSE: Tensor Norm Constraints for Unsupervised Contrastive Learning of Sentence Embeddings (AAAI 2025 · similarity 0.90)
- Narrowing the Gap between Supervised and Unsupervised Sentence Representation Learning with Large Language Model (AAAI 2024 · similarity 0.89)
- DenoSent: A Denoising Objective for Self-Supervised Sentence Representation Learning (AAAI 2024 · similarity 0.88)
- Repurposing Language Models into Embedding Models: Finding the Compute-Optimal Recipe (NeurIPS 2024 · similarity 0.87)
- Some Like It Small: Czech Semantic Embedding Models for Industry Applications (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.