Aditya Kamireddypalli, Joao Moura, Russell Buchanan, Matias Mattamala, Sethu Vijayakumar, and Subramanian Ramamoorthy, Attentive ContactFusion: Stochastic Poisson Surface Maps from Visual and Contact Sensing, Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2026), Pittsburgh, USA (2026). [pdf] [video] [citation]
Contact-rich insertion requires precise geometric alignment despite uncertainty in perception, geometry, and control.
When this uncertainty causes unintended contact, the resulting contact provides local information that can guide corrective motion. Sampling-based Model Predictive Control (SMPC) is well suited to this setting because it optimizes through forward simulation without requiring gradients through contact dynamics. However, standard SMPC methods typically use generic proposal distributions that do not exploit observed contact to determine where to search next.
We introduce ContactKernels, a contact-adaptive proposal kernel that uses geometry-conditioned wrench measurements to estimate contact location and local contact geometry. The estimated contact state directly adapts the proposal distribution: contact location biases the search toward corrective motions, while the contact normal shapes the sampling covariance. We evaluate ContactKernels with Predictive Sampling (PS) and Model Predictive Path Integral (MPPI) control across multiple insertion geometries in simulation and on a real robot.
Nikolaos Tsagkas, Andreas Sochopoulos, Duolikun Danier, Sethu Vijayakumar, Alexandros Kouris, Oisin Mac Aodha and Chris Xiaoxuan Lu, Attentive Feature Aggregation or: How Policies Learn to Stop Worrying about Robustness and Attend to Task-Relevant Visual Cues, Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2026), Pittsburgh, USA (2026). [pdf] [citation]
Standard pre-trained visual representations often encode irrelevant scene information, causing robot policies to fail when facing visual perturbations. We propose Attentive Feature Aggregation (AFA), a trainable pooling mechanism that learns to attend specifically to task-relevant regions (like objects and grippers), significantly outperforming standard pooling methods in out-of-domain scenarios without fine-tuning the vision backbone.