Research map / Reinforcement learning / Visual and representation RL

Decomposed Prompt Decision Transformer for Efficient Unseen Task Generalization

NeurIPS 2024 · Reinforcement learning › Visual and representation RL · approach: In-context and adaptive RL · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DPDT, a GPT2-initialized decision transformer that decomposes task prompts into a shared cross-task prompt and low-rank task-specific prompts, then adapts the shared prompt on unseen tasks via test-time feature-statistic alignment.

Problem

Multi-task offline RL with sequence models struggles with gradient conflicts across tasks and data hunger, and generalizing to unseen, unlabeled Meta-RL tasks with few parameters.

Approach

Decompose each task prompt as an element-wise product of a shared cross-task prompt and a low-rank task-specific prompt, distill per-task teacher prompts to separate the information, initialize from frozen GPT2-small, and adapt only the cross-task prompt at test time by aligning layer-wise token mean/variance with training statistics.

Key results

• In zero-shot Meta-RL, DPDT reaches an average return of 518.28 across tasks versus 373.88 for Prompt-DT and 354.04 for MT-ORL (Table 1).
• In few-shot settings, DPDT averages 484.66 versus 419.47 for Prompt-DT, and reaches -30.73 on Cheetah-vel with only 1.14% trainable parameters (Table 2).
• Ablation shows test-time adaptation on the cross-task prompt is essential: removing TTA drops MW MT50 from 1559.94 to 824.07 (Table 3).

Measured effects reported in the paper

Surprising findings

• Applying TTA to Prompt-DT degraded performance on some tasks, whereas it improved Soft-Prompt on all tasks.
• DPDT underperforms Prompt-DT on Ant-dir in zero-shot and few-shot settings, attributed to domain gap between language and control tasks.

Limitations

• Relies on large language model initialization, and domain differences between language and RL may limit gains (e.g., Ant-dir).
• Performance is sensitive to model size; larger GPT-MIDDLE degraded some tasks.
• Test-time adaptation uses sampled actions to build unlabeled test data, and the method's sensitivity to this construction is not deeply analyzed.

Experimental setup

Datasets: Cheetah-dir Cheetah-vel Ant-dir MetaWorld ML10 MetaWorld ML45 MetaWorld MT10 MetaWorld MT50

Benchmarks: Cheetah-dir Cheetah-vel Ant-dir MetaWorld ML10 MetaWorld ML45 MetaWorld MT10 MetaWorld MT50

Metrics: average episode accumulated return standard deviation over three runs

Baselines: MT-BC MT-ORL Soft-Prompt HDT Prompt-DT DPDT-WP Soft-Prompt-TTA Prompt-DT-TTA

Builds on: Decision Transformer Prompt-DT Prompt tuning LoRA Test-time prompt tuning

Closest papers by meaning

• Prompt Tuning Decision Transformers with Structured and Scalable Bandits (NeurIPS 2025 · similarity 0.90)
• Meta-DT: Offline Meta-RL as Conditional Sequence Modeling with World Model Disentanglement (NeurIPS 2024 · similarity 0.88)
• Mastering Massive Multi-Task Reinforcement Learning via Mixture-of-Expert Decision Transformer (ICML 2025 · similarity 0.88)
• DaCe-DT: Data-Centric Offline Multi-Task Reinforcement Learning via Adaptive Prompts and Trajectory Correction for Heterogeneous Tasks (NeurIPS 2026 · similarity 0.87)
• In-Context Decision Transformer: Reinforcement Learning via Hierarchical Chain-of-Thought (ICML 2024 · similarity 0.87)
• HarmoDT: Harmony Multi-Task Decision Transformer for Offline Reinforcement Learning (ICML 2024 · similarity 0.87)