Contemporary robotics focuses on avoiding contact with the environment, as such contact could potentially damage the robot or its surroundings. However, this approach doesn’t align with our vision for autonomous machines, which are meant to collaborate with humans. While present-day robots can lift and carry objects, there’s much progress needed for tasks like zipping up a jacket or peeling an apple.
Moreover, while robots can walk, jump, and perform backflips, these actions are predominantly executed on stable ground. Our aspiration is for robots to accomplish these tasks on varied surfaces, utilizing contact across their entire bodies, not just their feet or hands. The necessity for physical interaction is particularly evident in tasks involving robotic manipulation and legged locomotion. In both scenarios, a deep understanding of contact physics is essential for effective interaction with environment. We believe that in robotics, contact is not a bug but a feature.
In our research, we aim to challenge the current approach to robotics that avoids contact. We intend for robots to leverage contact for perception and action. Regarding perception, we focus on using vision and tactile sensing to provide robots with a detailed understanding of the environment’s physics. Our primary emphasis is on the sense of touch, as it directly measures physical parameters of objects the robot interacts with. Touch complements vision, offering information that is otherwise inaccessible through indirect visual sensing. Additionally, touch can guide the learning process of vision-based algorithms.
Our approach leverages environmental interaction to develop innovative methods that merge sensing modes, uncovering insights into interaction physics. We utilize cutting-edge tools, including machine learning, advanced computer vision, representation learning, and signal processing, with a specific emphasis on unsupervised techniques. In the action domain, understanding contact physics enhances robot task execution, avoiding worst-case assumptions for physical parameters (e.g., low friction for walking or grasping) to enable efficient interaction with the world.