Research map / Large language models / Reasoning and chain of thought

Meta-TTRL: A Metacognitive Framework for Self-Improving Test-Time Reinforcement Learning for T2I Generation in Unified Multimodal Models

NeurIPS 2026 · Large language models › Reasoning and chain of thought · approach: Test-time compute scaling

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Meta-TTRL, a test-time reinforcement learning framework in which a unified multimodal model uses its own rubric-based introspection as reward to update its T2I generator with GRPO at test time.

Problem

Test-time scaling for text-to-image generation in unified multimodal models yields only instance-level, ephemeral gains because parameters stay frozen; the paper asks whether test-time experience can drive parameter updates without external reward models.

Approach

A two-level metacognitive design: the same UMM acts as an object-level generator and a meta-level introspector that builds a binary-question rubric per prompt and scores candidate images by Yes/No token probabilities; these intrinsic rewards drive GRPO updates of the generator at test time.

Key results

• Qwen-Image DPG score rises from 88.32 to 89.00 with Meta-TTRL; BAGEL TIIF short rises from 71.65 to 75.98.
• Janus-Pro-7B gains are large on T2I-CompBench++ (e.g., color 52.02 to 79.33, 2D spatial 15.73 to 32.46).
• Meta-TTRL outperforms E-TTRL (Qwen3-VL-235B as introspector) on six of eight T2I-CompBench++ subdimensions for Janus-Pro-7B.

Surprising findings

• A stronger external introspector (Qwen3-VL-235B) gives less effective TTRL than the model's own intrinsic introspection, suggesting capacity-matched signals matter more than evaluator strength.
• Meta-TTRL outperforms RL leakage with UnifiedReward, a specially trained reward model, on TIIF and DPG benchmarks.

Limitations

• Requires access to model parameters, so it cannot be applied to closed-source models.
• Gains on stronger models such as Qwen-Image are moderate (e.g., DPG +0.68).

Experimental setup

Datasets: T2I-CompBench++ TIIF-Bench DPG-Bench

Benchmarks: TIIF-Bench T2I-CompBench++ DPG-Bench GenEval

Metrics: TIIF score (short/long) T2I-CompBench++ subdimension scores DPG score GenEval score

Baselines: Qwen-Image BAGEL Janus-Pro-7B E-TTRL RL Leakage (UnifiedReward) GIT-based rubric evaluation

Builds on: GRPO test-time reinforcement learning Nelson-Narens metacognition model

Closest papers by meaning

• TTRL: Test-Time Reinforcement Learning (NeurIPS 2025 · similarity 0.90)
• Tool Verification for Test-Time Reinforcement Learning (NeurIPS 2026 · similarity 0.88)
• SELF-HARMONY: LEARNING TO HARMONIZE SELF-SUPERVISION AND SELF-PLAY IN TEST-TIME REINFORCEMENT LEARNING (ICLR 2026 · similarity 0.88)
• Diving into Self-Evolving Training for Multimodal Reasoning (ICML 2025 · similarity 0.87)
• Optimizing Test-Time Compute via Meta Reinforcement Finetuning (ICML 2025 · similarity 0.87)
• MetaCARD: Meta-Reinforcement Learning with Task Uncertainty Feedback via Decoupled Context-Aware Reward and Dynamics Components (AAAI 2024 · similarity 0.87)