InsightCobot

Cooperative robots with sensitive fingers and efficient motion planning

01/05/2023 to 30/04/2026

Cooperative robots (cobots) are the next generation of autonomous robots for automation in production. They are designed to work flexibly together with humans. However, robots still lack skill in many manipulation tasks. This is due to the lack of haptic sensing, insufficient coupling of visual and tactile perception, and often inflexible motion planning for grasping and manipulation. In this research project, we demonstrate how novel tactile finger sensors, novel motion planning, and deep neural network-based object recognition and pose determination can enable efficient, sensitive grasping of diverse solid and soft objects. To this end, we bring together several key technologies recently developed by the partners: First, we are further developing new tactile finger sensors from the Martius group that enable accurate determination of the contact area and forces on the entire fingertip. These sensors are to be optimized for use with cobots. The visual recognition of the objects and their position and the combination of haptic and visual information will be done with deep neural networks. To make optimal use of the representation, grasping strategies will be learned that directly translate the visio-haptic information into motor control. For this, we use highly parallel physics simulations on graphics accelerators and techniques to transfer simulations to reality. A final building block is fast trajectory planning using geodesics to move the gripper without collision, taking into account obstacles in the workspace and constraints in the configuration space - a new method of robot motion planning from the Zell group. To demonstrate the capabilities, we will show examples of playing chess with real pieces and sorting vegetables. The developed methods should be applicable to cobots of German manufacturers (Franka Emika, KUKA, Festo, Neura Robotics) and find multiple applications in industry.