Research map / Reinforcement learning / Visual and representation RL

Text-to-Decision Agent: Offline Meta-Reinforcement Learning from Natural Language Supervision

NeurIPS 2025 · Reinforcement learning › Visual and representation RL · approach: Task-inference meta-RL · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes T2DA, which aligns text descriptions with dynamics-aware trajectory embeddings via CLIP-style contrastive pre-training, then trains text-conditioned diffusion or transformer policies for zero-shot offline meta-RL.

Problem

Offline meta-RL generalization to unseen tasks normally requires expert demonstrations or warmup exploration to infer task beliefs, which are costly or infeasible for new tasks; the paper asks whether natural language task descriptions can supply this supervision instead.

Approach

Pre-train a generalized world model to produce dynamics-aware trajectory embeddings, then fine-tune a text encoder (LoRA) with a symmetric contrastive loss to align task descriptions with these embeddings, and condition a diffuser or decision transformer on the aligned text embedding for zero-shot text-to-decision generation.

Key results

• On Mixed datasets, T2DA-T reaches 970.3 on Ant-Dir versus 786.9 for the best baseline (BAKU), and T2DA-D reaches 1376.4 on Meta-World versus 1187.6 for BAKU.
• Ablations on Mixed datasets show removing contrastive language-decision alignment (w/o align) or language supervision (w/o text) lowers Ant-Dir returns for T2DA-T from 963.4 to 792.3 and 242.6 respectively.
• LoRA fine-tuning of the CLIP text encoder uses under 1% of parameters (589,824 of 63.7M) and yields nearly identical performance to full fine-tuning.

Measured effects reported in the paper

Surprising findings

• Language-conditioned baselines BC-Z and BAKU outperform offline meta-RL and in-context RL baselines on harder environments such as Ant-Dir and Meta-World.
• Performance is nearly identical across CLIP, BERT, and T5 text encoders, suggesting the choice of language model matters little.

Limitations

• Trained only on lightweight datasets compared with large foundation models; scaling not demonstrated.
• Evaluated only on simulated MuJoCo and Meta-World tasks with a small number of test tasks (5 or 6).

Experimental setup

Datasets: Point-Robot offline datasets Cheetah-Vel offline datasets Ant-Dir offline datasets Meta-World offline datasets

Benchmarks: Point-Robot Cheetah-Vel Ant-Dir Meta-World

Metrics: zero-shot test return

Baselines: CSRO UNICORN AD BC-Z BAKU

Builds on: CLIP Decision Diffuser Decision Transformer Prompt-DT LoRA

Closest papers by meaning

• Meta-DT: Offline Meta-RL as Conditional Sequence Modeling with World Model Disentanglement (NeurIPS 2024 · similarity 0.88)
• RLZero: Direct Policy Inference from Language Without In-Domain Supervision (NeurIPS 2025 · similarity 0.88)
• AMAGO-2: Breaking the Multi-Task Barrier in Meta-Reinforcement Learning with Transformers (NeurIPS 2024 · similarity 0.88)
• Towards an Information Theoretic Framework of Context-Based Offline Meta-Reinforcement Learning (NeurIPS 2024 · similarity 0.87)
• Generalizable Task Representation Learning for Offline Meta-Reinforcement Learning with Data Limitations (AAAI 2024 · similarity 0.87)
• The Synergy of LLMs & RL Unlocks Offline Learning of Generalizable Language-Conditioned Policies with Low-fidelity Data (ICML 2025 · similarity 0.87)