Research map / Reinforcement learning / Visual and representation RL
Natural Language Instruction-following with Task-related Language Development and Translation
Jing-Cheng Pang, Xinyu Yang, Si-Hang Yang, Xiong-Hui Chen, Yang Yu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace natural-language policy input (outside-in learning) with learned task language TL (inside-out learning) | instruction-following policy · FrankaKitchen | success rate improvement (%): 63.2% | Improved |
| Replace natural-language policy input with learned task language TL | instruction-following policy · CLEVR-Robot | success rate improvement (%): 13.4% | Improved |
| Increase number of NL sentence patterns from 1 to 9 for TALAR | TALAR · CLEVR-Robot train | success rate (%): 100% -> 99.9% | No clear effect |
| Increase number of NL sentence patterns from 1 to 9 for One-hot baseline | One-hot · CLEVR-Robot train | success rate (%): 98.2% -> 86.5% | Hurt |
| Increase number of NL sentence patterns from 1 to 9 for BERT-binary baseline | BERT-binary · CLEVR-Robot train | success rate (%): 82.0% -> 64.0% | Hurt |
| Increase number of NL sentence patterns from 1 to 9 for BERT-continuous baseline | BERT-continuous · CLEVR-Robot train | success rate (%): 95.3% -> 60.7% | Hurt |
| Use TL instead of NL as high-level goal abstraction in hierarchical RL (vs HAL) | high-level policy · CLEVR-Robot and FrankaKitchen long-horizon tasks | sample efficiency (episodic reward) | Improved |
| Increase number of predicate networks Npn (2, 4, 6) and argument count Na (2, 3, 4) (ablation) | TALAR translator · FrankaKitchen (4M steps) | success rate (train/test) | Improved |
| Excessively large Npn and Na (ablation) | TALAR translator · FrankaKitchen | training efficiency | Hurt |
| Fine-tune BERT encoder on the task in OIL baseline | BERT-based baseline · FrankaKitchen t-SNE of NL encodings | uniqueness of encoding | No clear effect |
Closest papers by meaning
- Guiding Policies with Language via Meta-Learning (ICLR 2019 · similarity 0.91)
- RLZero: Direct Policy Inference from Language Without In-Domain Supervision (NeurIPS 2025 · similarity 0.90)
- From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Following (ICLR 2019 · similarity 0.89)
- The Synergy of LLMs & RL Unlocks Offline Learning of Generalizable Language-Conditioned Policies with Low-fidelity Data (ICML 2025 · similarity 0.88)
- DAIL: Beyond Task Ambiguity for Language-Conditioned Reinforcement Learning (NeurIPS 2025 · similarity 0.87)
- On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models (ICML 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
