Abstract: A method includes receiving, from a camera disposed on a robotic device, a two-dimensional (2D) image of a body of an actor and determining, for each respective keypoint of a first subset of a plurality of keypoints, 2D coordinates of the respective keypoint within the 2D image. The plurality of keypoints represent body locations. Each respective keypoint of the first subset is visible in the 2D image. The method also includes determining a second subset of the plurality of keypoints. Each respective keypoint of the second subset is not visible in the 2D image. The method further includes determining, by way of a machine learning model, an extent of engagement of the actor with the robotic device based on (i) the 2D coordinates of keypoints of the first subset and (ii) for each respective keypoint of the second subset, an indicator that the respective keypoint is not visible.

Abstract: A method includes receiving, from a camera disposed on a robotic device, a two-dimensional (2D) image of a body of an actor and determining, for each respective keypoint of a first subset of a plurality of keypoints, 2D coordinates of the respective keypoint within the 2D image. The plurality of keypoints represent body locations. Each respective keypoint of the first subset is visible in the 2D image. The method also includes determining a second subset of the plurality of keypoints. Each respective keypoint of the second subset is not visible in the 2D image. The method further includes determining, by way of a machine learning model, an extent of engagement of the actor with the robotic device based on (i) the 2D coordinates of keypoints of the first subset and (ii) for each respective keypoint of the second subset, an indicator that the respective keypoint is not visible.

Abstract: Methods, systems, and devices for managing robot resources are described. A robot receives from an application a request to reserve a particular set of physical resources of the robot. The robot then determines that each of the physical resources in the set are available to the application and, based on the determination, allocates exclusive use of the particular set of resources to the application by (i) generating a token corresponding to the set of resources, (ii) providing the token to the application, and (iii) updating token data that associates the token with the set of resources. The robot then controls access to the particular set of resources such that, while token data indicates that the token is valid, commands from applications that involve the set of resources are only executed when provided with the token corresponding to the allocation of access to the particular set of resources.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for receiving, by one or more non-real-time processors, data defining a light illumination pattern for a robotic device. Generating, by the one or more non-real-time processors and based on the data, a spline that represents the light illumination pattern, where a knot vector of the spline defines a timing profile of the light illumination pattern. Providing the spline to one or more real-time processors of the robotic system. Calculating, by the one or more real-time processors, an illumination value from the spline at each of a plurality of time steps. Controlling, by the one or more real-time processors, illumination of a lighting display of the robotic system in accordance with the illumination value of the spline at each respective time step.

Abstract: A method includes receiving, from a camera disposed on a robotic device, a two-dimensional (2D) image of a body of an actor and determining, for each respective keypoint of a first subset of a plurality of keypoints, 2D coordinates of the respective keypoint within the 2D image. The plurality of keypoints represent body locations. Each respective keypoint of the first subset is visible in the 2D image. The method also includes determining a second subset of the plurality of keypoints. Each respective keypoint of the second subset is not visible in the 2D image. The method further includes determining, by way of a machine learning model, an extent of engagement of the actor with the robotic device based on (i) the 2D coordinates of keypoints of the first subset and (ii) for each respective keypoint of the second subset, an indicator that the respective keypoint is not visible.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for receiving, by one or more non-real-time processors, data defining a light illumination pattern for a robotic device. Generating, by the one or more non-real-time processors and based on the data, a spline that represents the light illumination pattern, where a knot vector of the spline defines a timing profile of the light illumination pattern. Providing the spline to one or more real-time processors of the robotic system. Calculating, by the one or more real-time processors, an illumination value from the spline at each of a plurality of time steps. Controlling, by the one or more real-time processors, illumination of a lighting display of the robotic system in accordance with the illumination value of the spline at each respective time step.

Abstract: A method includes receiving, from a camera disposed on a robotic device, a two-dimensional (2D) image of a body of an actor and determining, for each respective keypoint of a first subset of a plurality of keypoints, 2D coordinates of the respective keypoint within the 2D image. The plurality of keypoints represent body locations. Each respective keypoint of the first subset is visible in the 2D image. The method also includes determining a second subset of the plurality of keypoints. Each respective keypoint of the second subset is not visible in the 2D image. The method further includes determining, by way of a machine learning model, an extent of engagement of the actor with the robotic device based on (i) the 2D coordinates of keypoints of the first subset and (ii) for each respective keypoint of the second subset, an indicator that the respective keypoint is not visible.

Abstract: Methods, systems, and apparatus, including computer-readable storage devices, for enhanced object discrimination by robots are described. In some implementations, a robot obtains image data from a camera of the robot, where the image data indicates a detected object. The robot classifies the object as having a particular object type based on the image data. The robot evaluates a current state of the object that includes a predetermined set of object state characteristics. The robot determines that a software application of the robot has indicated (i) an object type and (ii) one or more object state criteria that satisfy the current state of the detected object. The robot provides to the software application data indicating detection of the object and a position of the object.

Abstract: Methods, systems, and devices for managing robot resources are described. A robot receives from an application a request to reserve a particular set of physical resources of the robot. The robot then determines that each of the physical resources in the set are available to the application and, based on the determination, allocates exclusive use of the particular set of resources to the application by (i) generating a token corresponding to the set of resources, (ii) providing the token to the application, and (iii) updating token data that associates the token with the set of resources. The robot then controls access to the particular set of resources such that, while token data indicates that the token is valid, commands from applications that involve the set of resources are only executed when provided with the token corresponding to the allocation of access to the particular set of resources.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for receiving, by one or more non-real-time processors, data defining a light illumination pattern for a robotic device. Generating, by the one or more non-real-time processors and based on the data, a spline that represents the light illumination pattern, where a knot vector of the spline defines a timing profile of the light illumination pattern. Providing the spline to one or more real-time processors of the robotic system. Calculating, by the one or more real-time processors, an illumination value from the spline at each of a plurality of time steps. Controlling, by the one or more real-time processors, illumination of a lighting display of the robotic system in accordance with the illumination value of the spline at each respective time step.

Abstract: Methods, systems, and devices for managing robot resources are described. A robot receives from an application a request to reserve a particular set of physical resources of the robot. The robot then determines that each of the physical resources in the set are available to the application and, based on the determination, allocates exclusive use of the particular set of resources to the application by (i) generating a token corresponding to the set of resources, (ii) providing the token to the application, and (iii) updating token data that associates the token with the set of resources. The robot then controls access to the particular set of resources such that, while token data indicates that the token is valid, commands from applications that involve the set of resources are only executed when provided with the token corresponding to the allocation of access to the particular set of resources.

Abstract: Methods, systems, and apparatus, including computer-readable storage devices, for enhanced object discrimination by robots are described. In some implementations, a robot obtains image data from a camera of the robot, where the image data indicates a detected object. The robot classifies the object as having a particular object type based on the image data. The robot evaluates a current state of the object that includes a predetermined set of object state characteristics. The robot determines that a software application of the robot has indicated (i) an object type and (ii) one or more object state criteria that satisfy the current state of the detected object. The robot provides to the software application data indicating detection of the object and a position of the object.