Research map / Reinforcement learning / Visual and representation RL
Reinforcement Learning Control of a Physical Robot Device for Assisted Human Walking without a Simulator
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AIP, which pretrains an actor via behavior cloning on normative walking data (RIIV-processed IMU/MoCap states) and then adapts it online with dHDP actor-critic to personalize soft exosuit assistance on five participants without a simulator.
Problem
Controlling soft inflatable exosuits to assist human walking with RL is hard because there is no reliable dynamics model or simulator of human-exosuit interaction, data is limited, and real-time safety and fast personalization are required.
Approach
Combine a data-centric offline imitation policy (behavior cloning on normative gait data with reduced intra/inter-person variation normalization, RIIV) with online deterministic actor-critic (dHDP) adaptation directly on the physical exosuit, initialized from the offline policy.
Key results
- Online training reduced stage cost from about 0.94–1.42 to about 0.35–0.49 across the five participants.
- After online training, EMG effort was reduced relative to unassisted baseline for all participants, with reductions ranging from 5% to 48.2%.
- RIIV reduced offline action divergence versus DIRECT normalization (e.g., dA AD 0.0003 vs 0.259).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Adapt the offline imitation policy online with dHDP for each participant | AIP soft exosuit control · Level-ground walking, participants 1–5 | EMG effort: -14.4%; -48.2%; -36.8%; -5%; -8.5% | Helped |
| Process offline sensor data with RIIV rather than DIRECT normalization | Offline imitation learning policy · Offline normative walking policy data | action divergence: tA: 0.038 ± 0.002 -> 0.02 ± 0.002; dA: 0.259 ± 0.015 -> 0.0003 ± 0.007; tC: 0.21 ± 0.025 -> 0.02 ± 0.003 | Helped |
| Remove EMG effort from the control objective (ablation) | AIP online dHDP controller · Level-ground walking | EMG effort | Hurt |
| Remove kinematic error from the control objective (ablation) | AIP online dHDP controller · Level-ground walking | EMG effort | Hurt |
| Use only EMG effort or only kinematic error in the control objective instead of both (ablation) | AIP online dHDP controller · Level-ground walking | learning variance | Hurt |
| Use the full objective with both EMG effort and kinematic error (ablation) | AIP online dHDP controller · Level-ground walking | EMG effort | Helped |
| Apply AIP control on a 7-degree incline treadmill | AIP soft exosuit control · Incline walking, 7-degree treadmill | EMG effort | Helped |
Surprising findings
- Human adaptation alone without robot adaptation showed no consistent pattern, suggesting the gains come from human-robot co-adaptation.
- The offline policy alone did not reduce EMG effort for participants 1, 4 and 5.
Limitations
- Tested only for single-leg control; bilateral coordination not addressed.
- Reliance on data remains a persistent limitation; only healthy participants on level treadmill walking tested.
- Offline policy trained on only one participant's data.
Experimental setup
Datasets: IMU and EMG sensor data MoCap ground-truth gait data from one participant (offline)
Benchmarks: real-world human walking experiments on five participants (treadmill, 1 m/s) 7-degree incline treadmill walking
Metrics: stage cost peak knee error EMG effort action divergence convergence time
Baselines: DIRECT normalization for offline policy walking without exosuit assistance ablation without EMG term ablation without kinematic term frozen offline policy (no online adaptation)
Builds on: dHDP DDPG behavior cloning NFQCA
Closest papers by meaning
- FOSP: Fine-tuning Offline Safe Policy through World Models (ICLR 2025 · similarity 0.85)
- Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes (AAAI 2025 · similarity 0.85)
- RLIF: Interactive Imitation Learning as Reinforcement Learning (ICLR 2024 · similarity 0.85)
- CrossLoco: Human Motion Driven Control of Legged Robots via Guided Unsupervised Reinforcement Learning (ICLR 2024 · similarity 0.85)
- DUO: Diverse, Uncertain, On-Policy Query Generation and Selection for Reinforcement Learning from Human Feedback (AAAI 2025 · similarity 0.85)
- ReinFlow: Fine-tuning Flow Matching Policy with Online Reinforcement Learning (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.