Kejian Shi, Qingbiao Li, Chun Hei Lau, Tianyi Xia, Lei Song, Yonghao Long, Sethu Vijayakumar and Qi Dou, Execution-Level Correction for Surgical Robot Automation via Residual Policy Learning from Human Interventions, IEEE Robotics and Automation Letters (RAL) (2026). [DOI]
To be also presented at ICRA 2027 in Seoul.
Automation in robotic-assisted minimally invasive surgery (RMIS) demands millimeter-level execution precision. While imitation learning (IL) effectively captures long-horizon task structure, it falls short at deployment time in detecting and correcting localized execution drift at precision-critical stages such as grasping, handover, and tool-object contact. To address this, we present SuRResT (Surgical Robot Residual Transformer), a framework that learns execution-level corrections from human-in-the-loop (HIL) interventions collected during base policy rollouts. To extend correction coverage beyond explicit intervention timesteps, we augment the correction dataset through spatially-guided label propagation and a trajectory-aware deviation signal. A residual module is then trained on this dataset to selectively correct the frozen base policy at precision-critical states during deployment. We evaluate SuRResT across three surgical manipulation tasks in simulation and on the da Vinci Research Kit (dVRK) platform. Through targeted interventions concentrated at failure-prone states, SuRResT improves task success by up to 59.3 percentage points over the base IL policy in simulation. On the real-world dVRK hardware, SuRResT achieves 97.8\% success on Needle Grasp and Place and 93.3\% on Peg Transfer, substantially outperforming end-to-end retraining, finetuning, and data scaling baselines under matched human data-collection budgets.