Research map / Architectures and efficiency / Attention and transformers
Thoughtbubbles: an Unsupervised Method for Parallel Thinking in Latent Space
Houjun Liu, Shikhar Murty, Christopher D Manning, Róbert Csordás
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Thoughtbubbles forking instead of standard transformer at half the token budget (20B vs 40B) | 1.9B decoder LM · Validation perplexity | perplexity: 15.03 -> 12.68 | Improved |
| use Thoughtbubbles (k=2L) instead of baseline, 40B tokens each | 1.9B decoder LM · GSM8K | pass@1: 31.46 -> 32.10 | Improved |
| add Thoughtbubbles forking with k=4L instead of k=2L (ablation) | 772M on OpenWebText · OpenWebText perplexity | perplexity: 20.19 -> 19.74 | Improved |
| add adaptive forking versus non-adaptive duplicated filler tokens (Copy-3/Copy-5) (ablation) | 772M on OpenWebText · OpenWebText perplexity | perplexity: 20.90 -> 20.19 | Improved |
| use Thoughtbubbles instead of baseline on syntax task BLiMP | 150M-772M on OpenWebText/peS2o · BLiMP | accuracy: 80.4 vs Copy-3 81.2 (772M OpenWebText) | Mixed |
| use Thoughtbubbles at 319M instead of baseline at 772M | OpenWebText · OpenWebText perplexity | perplexity: 20.55 (319M ours k=2L) vs 21.22 (772M baseline) | Improved |
| use fixed forking budget instead of budget proportional to input size during autoregression (ablation) | Thoughtbubbles 772M · OpenWebText dev subset | perplexity: 21.18 -> 23.10 | Hurt |
| add dynamic forking budget in autoregression (ablation) | Thoughtbubbles 772M · OpenWebText dev subset | perplexity: 23.10 -> 21.18 | Improved |
| use Thoughtbubbles at 150M peS2o with k=2L | 150M on peS2o · peS2o LAMBADA | accuracy: 8.1 -> 5.0 | Hurt |
Closest papers by meaning
- Let Me Think! A Long Chain of Thought Can Be Worth Exponentially Many Short Ones (NeurIPS 2025 · similarity 0.88)
- Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach (NeurIPS 2025 · similarity 0.88)
- Towards Theoretical Understanding of Transformer Test-Time Computing: Investigation on In-Context Linear Regression (ICML 2026 · similarity 0.88)
- ThreadWeaver: Adaptive Threading for Efficient Parallel Reasoning in Language Models (ICML 2026 · similarity 0.87)
- Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokens (ICML 2026 · similarity 0.87)
- Energy-Based Transformers are Scalable Learners and Thinkers (ICLR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
