Abstract: A method for recording human motion performing a task with a tool and training a robot to perform the task, including the steps of performing tracker calibration without the robot present, performing teaching tool calibration with the robot present and placed at a predetermined location, and performing training without the robot present.

Abstract: Disclosed are a system and method for multimodal robot-human interaction using hand gestures and facial recognition for automated service robots. The system includes a robotic system with an integrated camera for capturing visual data, enabling hand gesture recognition and facial profiling. A control module processes this data to identify gestures and facial profiles, facilitating real-time interaction and decision-making. The gesture recognition module interprets hand gestures to control the robotic system's operations, while the facial recognition module identifies customers for personalized interactions. A communication interface supports interaction via text, voice, and video. The system is further enhanced by sensors for improved accuracy and a display module for visual feedback. The method involves capturing and processing visual data, interpreting gestures, and executing predefined actions, thereby streamlining tasks such as order and confirmation processes.

Abstract: Embodiments described herein provide various techniques and systems for teaching a multi-arm robot to imitate a two-handed beverage-preparation motion sequence performed by a human. In one aspect, a process for teaching a multi-arm robot to imitate the two-handed beverage-preparation motion sequence can begin by recording, in a teaching environment, two or more motion trajectories during the two-handed beverage-preparation motion sequence. The process then transforms the recorded two or more motion trajectories into a motion trajectory of an end effector of a first robotic arm. The process subsequently reproduces the two-handed beverage-preparation motion sequence by executing the transformed motion trajectory on an end effector of the first robotic arm.