AI Paper Atlas

Research map / Architectures and efficiency / Attention and transformers

Thoughtbubbles: an Unsupervised Method for Parallel Thinking in Latent Space

Houjun Liu, Shikhar Murty, Christopher D Manning, Róbert Csordás

ICML 2026 · Architectures and efficiency › Attention and transformers · approach: Efficient language transformers · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use Thoughtbubbles forking instead of standard transformer at half the token budget (20B vs 40B)1.9B decoder LM · Validation perplexityperplexity: 15.03 -> 12.68Improved
use Thoughtbubbles (k=2L) instead of baseline, 40B tokens each1.9B decoder LM · GSM8Kpass@1: 31.46 -> 32.10Improved
add Thoughtbubbles forking with k=4L instead of k=2L (ablation)772M on OpenWebText · OpenWebText perplexityperplexity: 20.19 -> 19.74Improved
add adaptive forking versus non-adaptive duplicated filler tokens (Copy-3/Copy-5) (ablation)772M on OpenWebText · OpenWebText perplexityperplexity: 20.90 -> 20.19Improved
use Thoughtbubbles instead of baseline on syntax task BLiMP150M-772M on OpenWebText/peS2o · BLiMPaccuracy: 80.4 vs Copy-3 81.2 (772M OpenWebText)Mixed
use Thoughtbubbles at 319M instead of baseline at 772MOpenWebText · OpenWebText perplexityperplexity: 20.55 (319M ours k=2L) vs 21.22 (772M baseline)Improved
use fixed forking budget instead of budget proportional to input size during autoregression (ablation)Thoughtbubbles 772M · OpenWebText dev subsetperplexity: 21.18 -> 23.10Hurt
add dynamic forking budget in autoregression (ablation)Thoughtbubbles 772M · OpenWebText dev subsetperplexity: 23.10 -> 21.18Improved
use Thoughtbubbles at 150M peS2o with k=2L150M on peS2o · peS2o LAMBADAaccuracy: 8.1 -> 5.0Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy