Research map / Large language models / Reasoning and chain of thought
Demo2Code: From Summarizing Demonstrations to Synthesizing Code via Extended Chain-of-Thought
Huaxiaoyue Wang, Gonzalo Gonzalez-Pumariega, Yash Sharma, Sanjiban Choudhury
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add demonstrations to language-only code generation (Demo2Code vs Lang2Code) | Lang2Code · Tabletop Manipulation overall | unit test pass rate: 0.19 -> 0.88 | Improved |
| use Demo2Code instead of Lang2Code | Lang2Code · Robotouille overall | unit test pass rate: 0.29 -> 0.59 | Improved |
| use Demo2Code instead of Lang2Code | Lang2Code · Robotouille overall | execution success: 0.64 -> 0.79 | Improved |
| use Demo2Code instead of Lang2Code | Lang2Code · Tabletop Manipulation overall | execution success: 1.00 -> 0.99 | Hurt |
| add language instruction to demonstration-only prompting (Demo2Code vs DemoNoLang2Code) | DemoNoLang2Code · Robotouille overall | execution success: 0.49 -> 0.79 | Improved |
| add language instruction to demonstration-only prompting | DemoNoLang2Code · Tabletop Manipulation overall | unit test pass rate: 0.81 -> 0.88 | Improved |
| use Demo2Code instead of DemoNoLang2Code | DemoNoLang2Code · Robotouille overall | Code BLEU: 0.71 -> 0.74 | No clear effect |
| use Demo2Code instead of Lang2Code | Lang2Code · EPIC-Kitchens | users with satisfied preference (of 7): 2 -> 5 | Improved |
| lengthen chain-of-thought from none to Full (extended CoT) (ablation) | Demo2Code · Robotouille short/medium/long horizon | unit test pass rate | Improved |
| use Full chain instead of 2-steps on long-horizon tasks (ablation) | Demo2Code · Robotouille long-horizon | unit test pass rate | Improved |
| use 1-step chain instead of Full on short-horizon tasks (ablation) | Demo2Code · Robotouille short-horizon | unit test pass rate | No clear effect |
| remove 10% of predicates or states from demonstrations (ablation) | Demo2Code · Robotouille overall | unit test pass rate | No clear effect |
Closest papers by meaning
- UniDomain: Pretraining a Unified PDDL Domain from Real-World Demonstrations for Generalizable Robot Task Planning (NeurIPS 2025 · similarity 0.86)
- GravMAD: Grounded Spatial Value Maps Guided Action Diffusion for Generalized 3D Manipulation (ICLR 2025 · similarity 0.86)
- Grounding Language Plans in Demonstrations Through Counterfactual Perturbations (ICLR 2024 · similarity 0.86)
- Text2Reward: Reward Shaping with Language Models for Reinforcement Learning (ICLR 2024 · similarity 0.85)
- Learning with Language-Guided State Abstractions (ICLR 2024 · similarity 0.85)
- GenSim: Generating Robotic Simulation Tasks via Large Language Models (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
