Research map / Label-efficient and robust learning / Self-supervised and contrastive
Contrastive Estimation Reveals Topic Posterior Information to Linear Models
Christopher Tosh, Akshay Krishnamurthy, Daniel Hsu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add document-level NCE contrastive representation instead of bag-of-words with a linear classifier | linear classifier on AG News · AG News (semi-supervised) | test accuracy | Improved |
| add document-level NCE representation instead of averaged word2vec embeddings | linear classifier on AG News · AG News (semi-supervised) | test accuracy | Improved |
| apply SVD dimensionality reduction to bag-of-words | BOW baseline · AG News (semi-supervised) | test accuracy | Hurt |
| use LDA topic posterior as the representation instead of NCE | linear classifier on AG News · AG News (semi-supervised) | test accuracy | Hurt |
| use bag-of-words with abundant labeled data | BOW baseline · AG News (large labeled set) | test accuracy | Mixed |
| increase contrastive network depth from one to two to three hidden layers (ablation) | NCE · AG News (semi-supervised) | test accuracy | Improved |
| lower the contrastive validation loss over training epochs (ablation) | NCE · AG News (1400 labeled examples) | test accuracy | Improved |
| increase Dirichlet hyperparameter alpha from 1 to 10 (topics more similar) (ablation) | NCE topic recovery · Simulated single-topic model (K=20) | MAP topic accuracy | Hurt |
| increase resampling rate from 0.1 to 1.0 (600K to 6M documents) (ablation) | NCE topic recovery · Simulated single-topic model (K=20) | MAP topic accuracy: 0.1 -> 1.0 | Improved |
| increase hidden layer width from 256 to 512 nodes (ablation) | NCE topic recovery · Simulated single-topic model (K=20) | MAP topic accuracy: 256 -> 512 | Improved |
Closest papers by meaning
- Topic Modeling as Multi-Objective Contrastive Optimization (ICLR 2024 · similarity 0.89)
- A Statistical Theory of Contrastive Learning via Approximate Sufficient Statistics (NeurIPS 2025 · similarity 0.86)
- Contrastive Learning with Simplicial Convolutional Networks for Short-Text Classification (ICML 2025 · similarity 0.85)
- Improved Graph Contrastive Learning for Short Text Classification (AAAI 2024 · similarity 0.85)
- A Simple Graph Contrastive Learning Framework for Short Text Classification (AAAI 2025 · similarity 0.84)
- Enhancing Visual Document Understanding with Contrastive Learning in Large Visual-Language Models (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
