Research map / Large language models / Reasoning and chain of thought

Reasoning Palette: Modulating Reasoning via Latent Contextualization for Controllable Exploration for (V)LMs

CVPR 2026 · Large language models › Reasoning and chain of thought · approach: Theory of autoregressive reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Reasoning Palette, a VAE-based latent modulation that decodes sampled latents into prefix token embeddings prepended to prompts, with brief SFT warm-up and latent-conditioned RL exploration for LLMs and VLMs.

Problem

Stochastic token-level sampling yields redundant reasoning paths with little high-level strategic diversity, limiting exploration in inference and RL training of (vision-)language models.

Approach

Learn a VAE over mean-pooled question-answer embeddings in the frozen token-embedding space; sampled latents are decoded into learnable prefix tokens prepended to the prompt, enabling pre-generative sampling over reasoning strategies, combined with SFT warm-up and a scheduled mix of latent-guided and unguided RL rollouts.

Key results

• On Qwen3-8B-Base with RLOO, Reasoning Palette (Two-Phase) raises average math accuracy from 59.91 to 62.28 (+2.37); linear decay reaches 63.00 (+3.09).
• On Qwen2.5-VL-3B referring expression comprehension, latent-guided + sampling reaches 87.53 pass@32 on RefCOCO vs 65.07 for baseline + sampling.
• On Qwen3-4B-Base, pass@32 on GSM8K rises from 52.9% to 85.3% with a single Gaussian-sampled prefix under greedy decoding (Qwen3-4B-Base motivating experiment).

Surprising findings

• Prepending a single random Gaussian-noise prefix under greedy decoding substantially boosts pass@k, suggesting gains come from strategic latent variation rather than token-level stochasticity.
• Naively injecting random noise often degrades performance due to misalignment with the model's embedding space.

Limitations

• Evaluation primarily on mathematical reasoning; VLM evaluation limited to referring expression comprehension.
• Gains over GRPO/RLOO are modest on several benchmarks (e.g., +1 to +3 average points), and some cells show regressions (e.g., AMC23 under linear decay for Qwen3-4B-Base GRPO).
• Hardware, training compute, and hyperparameters for RL are not reported in the provided text.

Experimental setup

Datasets: MetaMathQA competition_math PRM800K ShareGPT_Vicuna_unfiltered CodeParrot DeepMath RefCOCO RefCOCO+ RefCOCOg

Benchmarks: MATH500 OlympiadBench AMC23 GSM8K MinervaMath RefCOCO RefCOCO+ RefCOCOg

Metrics: pass@1 pass@8 pass@32 accuracy (IoU>=0.5)

Baselines: GRPO RLOO

Builds on: GRPO RLOO PPO VLM-R1 prefix-tuning

Closest papers by meaning

• Reasoning via Test-Time Instance-Level Policy Gradient in Latent Space (NeurIPS 2026 · similarity 0.89)
• Thinking on the Fly: Test-Time Reasoning Enhancement via Latent Thought Policy Optimization (ICLR 2026 · similarity 0.89)
• Hybrid Latent Reasoning via Reinforcement Learning (NeurIPS 2025 · similarity 0.89)
• Latent Visual Reasoning (ICLR 2026 · similarity 0.89)
• LoongRL: Reinforcement Learning for Advanced Reasoning over Long Contexts (ICLR 2026 · similarity 0.88)
• Learning to Disentangle Latent Reasoning Rules with Language VAEs: A Systematic Study (AAAI 2026 · similarity 0.88)