The goal of the project is to develop a robotic system that can accurately hand over objects to humans, ensuring precise placement without re-gripping. The project addresses three main challenges: achieving effective scene understanding and situation awareness, accurately recognizing user intentions and high-level reasoning, and teaching and executing collaborative tasks. By focusing on these areas, the project seeks to improve intuitive robot instruction and responsive collaboration in medical contexts, ultimately contributing to more efficient workflows and addressing labor shortages in sectors like healthcare.

The project plan focuses on developing methods to determine which objects to hand over, how to place them in the user’s hand, and how to execute these tasks effectively. Hand tracking enables the robot to understand and respond to human motion, ensuring precise object handovers and lets the robot accurately place the tool in their hand without the need for re-gripping. Tool tracking, on the other hand, allows the robot to accurately identify and manipulate these specific tools during collaborative tasks, ensuring that each tool is handed over in the correct orientation and position for immediate use.

By integrating high-level knowledge and explicit commands, the system can predict what tool to hand over and when, based on the context of the task and the workflow. This involves using symbolic domain knowledge and sensor information to understand the user’s intentions and make informed decisions. The system aims to balance personalization with general solutions, learning individual user preferences for gestures, speed, and force parameters, while also maintaining a level of generality that allows it to adapt to different users and scenarios.

Further steps in the project include evaluating different motion models to enhance the robot’s performance in various scenarios. This will involve testing and refining the robot’s ability to execute smooth and precise movements, ensuring that it can handle unexpected deviations in the workflow and still perform accurate handovers. By focusing on these aspects, the project aims to improve intuitive robot instruction and responsive collaboration in both medical and industrial contexts, ultimately contributing to more efficient workflows.