AI Paper Atlas

Research map / Probabilistic and scientific modeling / Variational and generative sampling

Variational Reasoning for Language Models

Xiangxin Zhou, Zichen Liu, Haonan Wang, Chao Du, Min Lin, Chongxuan Li, Liang Wang, Tianyu Pang

ICLR 2026 · Probabilistic and scientific modeling › Variational and generative sampling · approach: Amortized and online variational inference

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
condition the variational posterior on an answer hint y' (ablation)Variational reasoning, Qwen3-4B-Base · math suiteaverage accuracy: 48.18 -> 55.72Improved
condition the variational posterior on an answer hint y' (ablation)Variational reasoning, Qwen3-4B-Base · other suiteaverage accuracy: 37.80 -> 46.12Improved
use accuracy-based estimator instead of naive likelihood for pi(Y|x,z) (ablation)Variational reasoning, Qwen2.5-7B-Instruct 1K · math suiteaverage accuracy: 43.01 -> 45.41Improved
use geometric-mean token likelihood instead of naive likelihood (ablation)Variational reasoning, Qwen2.5-7B-Instruct 1K · other suiteaverage accuracy: 37.35 -> 39.42Improved
accuracy-based vs GML estimator (ablation)Variational reasoning · four model scalesaverage accuracy: 55.72 vs 55.23 (Qwen3-4B math)No clear effect
increase number of sampled traces K from 1 to 32 (ablation)Variational reasoning, 1K data · reasoning benchmarksaccuracyImproved
variational reasoning vs distillation on same dataQwen3-4B-Base · math suiteaverage accuracy: 51.35 -> 55.72Improved
variational reasoning vs distillation on same dataQwen3-8B-Base · other suiteaverage accuracy: 49.46 -> 54.69Improved
variational reasoning vs RLTQwen2.5-32B-Instruct · math suiteaverage accuracy: 70.43 -> 72.01Improved
switch prompt template A to B for the posteriorVariational reasoning · reasoning benchmarksaverage accuracyNo clear effect
add variational weighting to SFTQwen3-4B/8B-Base · trainingloss and gradient normImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy