Research map / Reinforcement learning / Visual and representation RL
ASID: Active Exploration for System Identification in Robotic Manipulation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ASID, which learns a Fisher-information-maximizing exploration policy in simulation, runs it for one real episode, identifies simulator parameters via optimization-based system identification, and trains downstream policies in simulation for zero-shot sim-to-real transfer.
Problem
Sim-to-real transfer requires highly accurate simulators, which demand substantial manual effort to specify physical parameters; policies trained in an uninformed or randomized simulator often fail in the real world when unknown parameters (mass, friction, articulation) matter.
Approach
Decouple exploration from exploitation: train an exploration policy in simulation to maximize the Fisher information (via finite-difference parameter sensitivity of the nominal dynamics, with domain randomization over parameters), execute it once in the real world, then identify simulator parameters from that single trajectory and solve the downstream task in the refined simulator.
Key results
- In simulated rod balancing, ASID + SysID achieves 0.00 tilt for left and right inertia and 0.72 for middle, versus 12.44 to 26.69 for random exploration and DR.
- In simulated sphere striking with unknown friction, ASID + SysID reaches 28.00% success versus 10.62% for random exploration and 8.75% for DR.
- On real-world rod balancing, ASID solves the task 6/9 times, while DR without environment interaction fails; on real shuffleboard, ASID lands the puck in the target zone 7/10 times versus 3/10 for DR.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Fisher-information-targeted exploration instead of random exploration | ASID · Sphere Striking — Friction ~[1.1, 1.5] | success rate (%): 10.62\pm4.3 -> 28.00\pm9.7 | Helped |
| replace optimization-based SysID with a learned estimator (ablation) | ASID · Sphere Striking — Friction ~[1.1, 1.5] | success rate (%): 11.00\pm5.2 -> 28.00\pm9.7 | Hurt |
| replace optimization-based SysID with a learned estimator (ablation) | ASID · Rod Balancing — Inertia (left) | tilt angle (degree): 0.00\pm0.0 -> 17.73\pm13.1 | Hurt |
| replace optimization-based SysID with a learned estimator (ablation) | ASID · Rod Balancing — Inertia (middle) | tilt angle (degree): 0.72\pm1.0 -> 4.65\pm5.1 | Hurt |
| replace optimization-based SysID with a learned estimator (ablation) | ASID · Rod Balancing — Inertia (right) | tilt angle (degree): 0.00\pm0.0 -> 9.99\pm6.8 | Hurt |
| use the ASID exploration and identification pipeline rather than domain randomization without environment interaction | Real-world Rod Balancing · Real-world Rod Balancing | solves: 6/9 vs. fails to solve it at all | Helped |
| add a dedicated exploration phase to adapt the simulator to current surface friction | Real-world Shuffleboard · Real-world Shuffleboard | successes: 3/10 -> 7/10 | Helped |
| interact with the laptop during exploration rather than use naïve baseline exploration | ASID · Laptop articulation | interaction frequency: 20% -> 80% | Helped |
Surprising findings
- A learned forward dynamics model trained on exploration data hallucinates sphere movement on out-of-distribution trajectories, whereas the identified simulator extrapolates correctly.
- Learned end-to-end estimators for system identification fail to generalize to unseen trajectories and perform much worse than optimization-based system identification.
Limitations
- Ambiguities arise when the center of mass is near the middle of the rod, causing identification failures and sim inaccuracy in real rod balancing.
- Assumes a known parametric dynamics family and a sufficiently accurate simulator with a known structure.
- Real-world experiments use color-thresholded point clouds from two cameras for pose estimation, with limited trial counts (9 and 10).
Experimental setup
Datasets: simulated trajectories from sphere, articulation, and rod balancing environments one real-world episode per real task
Benchmarks: rod balancing (simulation) sphere striking (simulation) multi-friction sphere exploration articulation (simulation) real-world rod balancing real-world shuffleboard
Metrics: tilt angle (degrees) success rate (%) solve count out of trials
Baselines: Random exploration MaxMI MAX Domain randomization ASID + estimator
Builds on: Fisher information experiment design domain randomization REPS PPO Ditto
Closest papers by meaning
- Rapidly Adapting Policies to the Real-World via Simulation-Guided Fine-Tuning (ICLR 2025 · similarity 0.89)
- Overcoming the Sim-to-Real Gap: Leveraging Simulation to Learn to Explore for Real-World RL (NeurIPS 2024 · similarity 0.88)
- Generalizable Domain Adaptation for Sim-and-Real Policy Co-Training (NeurIPS 2025 · similarity 0.87)
- Multi-Stage Manipulation with Demonstration-Augmented Reward, Policy, and World Model Learning (ICML 2025 · similarity 0.87)
- Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulation (ICLR 2025 · similarity 0.86)
- RLIF: Interactive Imitation Learning as Reinforcement Learning (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.