This one-page faculty research overview argues that robots should use physical interaction both to accomplish tasks and to obtain information. Its organizing idea is to balance computation with action in an ongoing perception–planning–interaction loop. It introduces a research agenda and points to prior studies, without presenting a new experiment or benchmark.
Imitation learning
Demonstrations transfer useful interaction skills to robots. ScrewMimic [1] learns bimanual tasks from a single human video by representing relative hand motion in screw space. Teleoperation and human-in-the-loop learning provide demonstrations suited to a robot’s embodiment [6], [5].
Interactive navigation
Interactive navigation includes moving obstacles, opening doors, and operating buttons when a collision-free route alone cannot solve the task. The overview points to hierarchical reinforcement-learning methods for mobile manipulators, including HRL4IN[4] and ReLMoGen [9].
Perceiving through interaction
Contact can reveal task-relevant information that passive sensing misses. Examples include moving clutter to find a target [2] and estimating articulated-object structure through interaction [7], [8]. The motivation draws on ecological accounts of perception [3].
Continual learning outlook
The proposed next step is continual learning: progressively acquire skills through imitation and exploration while improving representations learned from physical interactions. This is a future direction, rather than a result established in this overview.
9 references
1 Bahety, A.; et al. 2024. ScrewMimic: Bimanual Imitation from Human Videos with Screw Space Projection. In Robotics: Science and Systems (RSS). Best Student Paper Finalist.
2 Danielczuk, M.; et al. 2019. Mechanical search: Multi-step retrieval of a target object occluded by clutter. In IEEE International Conference on Robotics and Automation (ICRA).
4 Li, C.; et al. 2020. Hrl4in: Hierarchical reinforcement learning for interactive navigation with mobile manipulators. In Conference on Robot Learning (CoRL), 603–616. PMLR.
5 Liu, H.; et al. 2024. Model-based runtime monitoring with interactive imitation learning. In IEEE International Conference on Robotics and Automation (ICRA), 4154–4161. IEEE.
7 Martín-Martín, R.; and Brock, O. 2014. Online interactive perception of articulated objects with multi-level recursive estimation based on task-specific priors. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2494–2501. IEEE. Best Paper, Best Student Paper and Best Cognitive Paper Finalist.
8 Martín-Martín, R.; and Brock, O. 2017. Building Kinematic and Dynamic Models of Articulated Objects with Multi-Modal Interactive Perception. In AAAI Spring Symposium Series. AAAI Pubs.
9 Xia, F.; et al. 2021. ReLMoGen: Leveraging motion generation in reinforcement learning for mobile manipulation. In IEEE International Conference on Robotics and Automation (ICRA).