AI Paper Atlas

Research map / Label-efficient and robust learning / Self-supervised and contrastive

Contrastive Estimation Reveals Topic Posterior Information to Linear Models

Christopher Tosh, Akshay Krishnamurthy, Daniel Hsu

JMLR 2021 · Label-efficient and robust learning › Self-supervised and contrastive · approach: Inductive bias and minimal representations · 25 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add document-level NCE contrastive representation instead of bag-of-words with a linear classifierlinear classifier on AG News · AG News (semi-supervised)test accuracyImproved
add document-level NCE representation instead of averaged word2vec embeddingslinear classifier on AG News · AG News (semi-supervised)test accuracyImproved
apply SVD dimensionality reduction to bag-of-wordsBOW baseline · AG News (semi-supervised)test accuracyHurt
use LDA topic posterior as the representation instead of NCElinear classifier on AG News · AG News (semi-supervised)test accuracyHurt
use bag-of-words with abundant labeled dataBOW baseline · AG News (large labeled set)test accuracyMixed
increase contrastive network depth from one to two to three hidden layers (ablation)NCE · AG News (semi-supervised)test accuracyImproved
lower the contrastive validation loss over training epochs (ablation)NCE · AG News (1400 labeled examples)test accuracyImproved
increase Dirichlet hyperparameter alpha from 1 to 10 (topics more similar) (ablation)NCE topic recovery · Simulated single-topic model (K=20)MAP topic accuracyHurt
increase resampling rate from 0.1 to 1.0 (600K to 6M documents) (ablation)NCE topic recovery · Simulated single-topic model (K=20)MAP topic accuracy: 0.1 -> 1.0Improved
increase hidden layer width from 256 to 512 nodes (ablation)NCE topic recovery · Simulated single-topic model (K=20)MAP topic accuracy: 256 -> 512Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy