Research map / Reinforcement learning / Visual and representation RL
Ask Your Humans: Using Human Instructions to Improve Generalization in Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use human-language high-level instructions with a language-conditioned policy | Crafting agent on unseen tasks · Zero-shot crafts (35 unseen), Overall-AVG | accuracy: 55 -> 69 | Helped |
| Use human-language high-level instructions with a language-conditioned policy | Crafting agent on unseen tasks · Zero-shot crafts, 5-AVG | accuracy: 0 -> 10 | Helped |
| Add sparse-reward RL fine-tuning after human demonstration pretraining | IL w/ Generative Language · Standard setting, difficult crafting tasks | accuracy | Helped |
| Add human demonstrations to RL training | RL crafting agent · Standard setting, tasks with 3 or more steps | accuracy | Helped |
| Use only 25% of the human demonstrations instead of the full dataset (ablation) | Ours · Standard setting, hardest tasks | accuracy | Hurt |
| Replace generated language with discriminative selection from a fixed instruction set (ablation) | IL language model · Standard setting | interpretability | Hurt |
| Remove the state encoding from the policy input (ablation) | Ours · Standard setting, 1-step task | accuracy | Hurt |
| Use state reconstruction as the high-level representation instead of language (ablation) | Crafting agent · Zero-shot crafts (35 unseen), Overall-AVG | accuracy: 30 -> 69 | Hurt |
| Use state prediction as the high-level representation instead of language (ablation) | Crafting agent · Zero-shot crafts (35 unseen), Overall-AVG | accuracy: 22 -> 69 | Hurt |
| Train with demonstrations but omit natural-language instructions (ablation) | IL+RL · Zero-shot crafts (35 unseen), Overall-AVG | accuracy: 55 -> 69 | Hurt |
Closest papers by meaning
- Guiding Policies with Language via Meta-Learning (ICLR 2019 · similarity 0.90)
- Subwords as Skills: Tokenization for Sparse-Reward Reinforcement Learning (NeurIPS 2024 · similarity 0.89)
- Prompted Policy Search: Reinforcement Learning through Linguistic and Numerical Reasoning in LLMs (NeurIPS 2025 · similarity 0.88)
- Generalizing Verifiable Instruction Following (NeurIPS 2025 · similarity 0.88)
- Ground-Compose-Reinforce: Grounding Language in Agentic Behaviours using Limited Data (NeurIPS 2025 · similarity 0.88)
- Skill-SD: Skill-Conditioned Self-Distillation for Multi-turn LLM Agents (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.