Research map / Vision-language and multimodal / Robot action and navigation
DAIL: Beyond Task Ambiguity for Language-Conditioned Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DAIL, an offline language-conditioned RL method combining distributional value learning with a trajectory-instruction contrastive alignment loss, evaluated on BabyAI SynthLoc and ALFRED GOTO.
Problem
Task ambiguity in language-conditioned RL: many linguistic instructions map to overlapping tasks, so expectation-based value estimates become indistinguishable and performance degrades as the number of instructions grows.
Approach
Combine a distributional (categorical) value estimator, argued to be more task-discriminative and sample-efficient than expected values, with an InfoNCE-style trajectory-instruction mutual information alignment loss, plus a CQL offline regularizer.
Key results
- On BabyAI out-of-distribution tasks, DAIL reaches 81.7% overall success vs 72.6% for CQL and 71.2% for GRIF.
- On ALFRED GOTO validation, DAIL achieves 56.8% SR vs 50.4% for CQL and 52.0% for IQL.
- Ablation on BabyAI: removing alignment gives 75.1% OOD overall and removing distributional gives 77.3%, versus 81.7% for full DAIL.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use DAIL instead of the strongest reported baseline | language-conditioned offline RL · BabyAI SynthLoc OOD All | success rate: 74.1\textpm0.7 -> 81.7\textpm1.3 | Helped |
| use DAIL instead of the strongest reported baseline | language-conditioned offline RL · ALFRED Validation | SR: 52.0\textpm3.0 -> 56.8\textpm2.1 | Helped |
| use DAIL instead of the strongest reported baseline | language-conditioned offline RL · ALFRED Validation | PLW SR: 47.0\textpm3.2 -> 50.3\textpm2.4 | Helped |
| add distributional policy (ablation) | DAIL · BabyAI SynthLoc In Distribution All | success rate: 83.3\textpm0.2 -> 89.2\textpm0.5 | Helped |
| add distributional policy (ablation) | DAIL · BabyAI SynthLoc OOD All | success rate: 77.3\textpm0.8 -> 81.7\textpm1.3 | Helped |
| add trajectory-wise semantic alignment (ablation) | DAIL · BabyAI SynthLoc In Distribution All | success rate: 82.2\textpm1.3 -> 89.2\textpm0.5 | Helped |
| add trajectory-wise semantic alignment (ablation) | DAIL · BabyAI SynthLoc OOD All | success rate: 75.1\textpm1.6 -> 81.7\textpm1.3 | Helped |
| set alignment-loss weight to 0.2 rather than 1 (ablation) | DAIL · BabyAI tasks | performance | No clear effect |
| use an alignment-loss weight that is too small or too large (ablation) | DAIL · BabyAI tasks | performance | Hurt |
Surprising findings
- Vanilla offline RL methods (CQL, IQL) underperform imitation baselines such as GCBC on BabyAI, which the authors attribute to task ambiguity.
- A toy experiment shows doubling CQL's parameters does not resolve the drop in success as the number of instructions grows.
Limitations
- Not tested in real-world scenes.
- Theoretical advantage of distributional RL is derived only for the offline setting.
- Gains are reported on simulated benchmarks (BabyAI, ALFRED) only.
Experimental setup
Datasets: BabyAI SynthLoc ALFRED
Benchmarks: BabyAI SynthLoc ALFRED GOTO
Metrics: success rate path-length weighted success rate silhouette score
Baselines: CQL IQL GCBC BC-Z GRIF
Builds on: CQL distributional RL InfoNCE BabyAI ALFRED
Closest papers by meaning
- Ground-Compose-Reinforce: Grounding Language in Agentic Behaviours using Limited Data (NeurIPS 2025 · similarity 0.87)
- Making Expert Reasoning Learnable with Self-Distillation (ICML 2026 · similarity 0.87)
- FLAME : Factuality-Aware Alignment for Large Language Models (NeurIPS 2024 · similarity 0.86)
- BRAIn: Bayesian Reward-conditioned Amortized Inference for natural language generation from feedback (ICML 2024 · similarity 0.86)
- LLM Alignment--Utility Asymmetry under Semantic-Preserving Transformations (NeurIPS 2026 · similarity 0.86)
- Linear Alignment: A Closed-form Solution for Aligning Human Preferences without Tuning and Feedback (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.