Research map / Reinforcement learning / Imitation learning
SurgicAI: A Hierarchical Platform for Fine-Grained Surgical Policy Learning and Benchmarking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces SurgicAI, a simulation platform on the SRC/AMBF suturing environment with a hierarchical HLP/LLP framework, expert demonstration pipeline, and benchmarks of RL and IL algorithms on five suturing subtasks.
Problem
Automating complex, multi-stage surgical suturing with RL and IL requires realistic, dexterous simulation environments with deformable threads and standardized benchmarks; existing platforms only cover simple, non-dexterous tasks.
Approach
Combines a deformable-thread AMBF/SRC suturing simulator compatible with dVRK with a hierarchical framework (discrete high-level policy selecting among five subtask low-level policies), standardized demonstration collection, and benchmarks of many RL/IL algorithms with BC and Q-filter variants.
Key results
- TD3+HER+BC reached 0.96 to 1.00 success on grasping, placing, inserting, handoff and pullout subtasks with sparse rewards, while PPO and DDPG were near zero on most subtasks.
- The hierarchical framework achieved 0.52 to 0.88 success on sequential subtask chains versus 0.00 for TD3+HER, and over 50% success on the full suturing procedure.
- Pretrained visual encoders (R3M, CLIP, ImageNet) outperformed a randomly initialized ResNet-50 in image-based imitation learning; the best handoff success was 0.98 with R3M back camera.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| decompose sequential tasks into a hierarchy rather than use a single policy | SurgicAI suturing policies · sequential subtasks | success rate: 0.64 -> 0.88; 0.40 -> 0.72; 0.24 -> 0.60; 0.15 -> 0.52 | Helped |
| add behavior cloning to TD3+HER (ablation) | TD3+HER · sequential subtasks | success rate: 0.00 -> 0.64; 0.00 -> 0.40; 0.00 -> 0.24; 0.00 -> 0.15 | Helped |
| add the RL loss to behavior cloning | BC policies · Approaching | trajectory length: approximately 5 mm | Helped |
| add the RL loss to behavior cloning | BC policies · Approaching | time steps: 10 steps | Helped |
| use pretrained visual representations instead of a randomly initialized encoder | image-based imitation policies · surgical subtasks | success rate | Helped |
| use the back camera rather than the front camera | image-based imitation policies · Handoff | success rate: 0.80±0.04 -> 0.98±0.02 | Helped |
| increase expert trajectories from 10 to 30 episodes (ablation) | TD3+HER+BC · Approaching | success rate: 0.61 -> 0.90 | Helped |
| increase BC loss weight from 0.2 to 0.8 (ablation) | TD3+HER+BC · Placing, 30 expert episodes | success rate: 0.84 -> 1.00 | Helped |
| increase BC loss weight from 0.2 to 0.8 (ablation) | TD3+HER+BC · Inserting, 30 expert episodes | success rate: 0.80 -> 0.95 | Helped |
| increase BC loss weight from 0.5 to 0.8 (ablation) | TD3+HER+BC · Approaching, 10 expert episodes | success rate: 0.61 -> 0.60 | No clear effect |
Surprising findings
- Pure online RL (PPO, DDPG) with dense or sparse rewards achieved near-zero success on most subtasks, and AWAC and CalQL scored zero on some tasks.
- BC alone reached comparable subtask success to TD3+HER+BC, with the RL loss mainly improving trajectory efficiency.
Limitations
- Image-based policies show suboptimal and unstable success rates, sensitive to camera viewpoint.
- Current policies are Markovian single-step methods, inadequate for temporal dependencies such as pauses or corrections.
- Simulator is the 2022-2023 AccelNet SRC version; no sim-to-real transfer demonstrated.
Experimental setup
Datasets: human teleoperation demonstrations (dVRK MTM) heuristic DMP/LWR trajectories
Benchmarks: SurgicAI suturing subtasks (grasping, placing, inserting, handoff, pullout)
Metrics: success rate trajectory length (mm) time steps
Baselines: PPO DDPG SAC TD3 TD3+HER TD3+BC BC AWAC BCQ IQL CalQL R3M
Builds on: Surgical Robotics Challenge simulator AMBF TD3+HER Gymnasium Stable Baselines3
Closest papers by meaning
- SurgVista: Long-Horizon Surgical World Modeling with Plausible Instrument-Tissue Dynamics (NeurIPS 2026 · similarity 0.87)
- SutureBot: A Precision Framework & Benchmark For Autonomous End-to-End Suturing (NeurIPS 2025 · similarity 0.87)
- SonoGym: High Performance Simulation for Challenging Surgical Tasks with Robotic Ultrasound (NeurIPS 2025 · similarity 0.85)
- ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning (CVPR 2025 · similarity 0.84)
- DexFlyWheel: A Scalable and Self-improving Data Generation Framework for Dexterous Manipulation (NeurIPS 2025 · similarity 0.83)
- BAKU: An Efficient Transformer for Multi-Task Policy Learning (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.