Abstract: A robotic gripping device is provided. The robotic gripping device includes a palm and a plurality of digits coupled to the palm. The robotic gripping device also includes a time-of-flight sensor arranged on the palm such that the time-of-flight sensor is configured to generate time-of-flight distance data in a direction between the plurality of digits. The robotic gripping device additionally includes an infrared camera, including an infrared illumination source, where the infrared camera is arranged on the palm such that the infrared camera is configured to generate grayscale image data in the direction between the plurality of digits.

Abstract: In one embodiment, a method includes receiving, from a first sensor on a robot, first sensor data indicative of an environment of the robot. The method also includes identifying, based on the first sensor data, an object of an object type in the environment of the robot, where the object type is associated with a classifier that takes sensor data from a predetermined pose relative to the object as input. The method further includes causing the robot to position a second sensor on the robot at the predetermined pose relative to the object. The method additionally includes receiving, from the second sensor, second sensor data indicative of the object while the second sensor is positioned at the predetermined pose relative to the object. The method further includes determining, by inputting the second sensor data into the classifier, a property of the object.

Abstract: A robotic gripping device is provided. The robotic gripping device includes a palm and a plurality of digits coupled to the palm. The robotic gripping device also includes a time-of-flight sensor arranged on the palm such that the time-of-flight sensor is configured to generate time-of-flight distance data in a direction between the plurality of digits. The robotic gripping device additionally includes an infrared camera, including an infrared illumination source, where the infrared camera is arranged on the palm such that the infrared camera is configured to generate grayscale image data in the direction between the plurality of digits.

Abstract: A robotic gripping device is provided. The robotic gripping device includes a palm and a plurality of digits coupled to the palm. The robotic gripping device also includes a time-of-flight sensor arranged on the palm such that the time-of-flight sensor is configured to generate time-of-flight distance data in a direction between the plurality of digits. The robotic gripping device additionally includes an infrared camera, including an infrared illumination source, where the infrared camera is arranged on the palm such that the infrared camera is configured to generate grayscale image data in the direction between the plurality of digits.

Abstract: A method is provided that includes controlling a robotic gripping device to cause a plurality of digits of the robotic gripping device to move towards each other in an attempt to grasp an object. The method also includes receiving, from at least one non-contact sensor on the robotic gripping device, first sensor data indicative of a region between the plurality of digits of the robotic gripping device. The method further includes receiving, from the at least one non-contact sensor on the robotic gripping device, second sensor data indicative of the region between the plurality of digits of the robotic gripping device, where the second sensor data is based on a different sensing modality than the first sensor data. The method additionally includes determining, using an object-in-hand classifier that takes as input the first sensor data and the second sensor data, a result of the attempt to grasp the object.

Abstract: Methods, apparatus, and computer readable media applicable to balancing robots. Some implementations are directed to maintaining a given end effector pose (relative to a world frame) of an end effector of a balancing robot when there is a disturbance to a balancing base of the balancing robot. Some implementations are additionally or alternatively directed to transitioning a balancing robot from a fallen configuration to a balanced configuration. Some implementations are additionally or alternatively directed to mitigating the risk that a balancing robot will fall when interacting with actuable environmental objects (e.g., doors) and/or to lessen the disturbance to a balancing base when interacting with actuable environmental objects.

Abstract: Generating and utilizing action image(s) that represent a candidate pose (e.g., a candidate end effector pose), in determining whether to utilize the candidate pose in performance of a robotic task. The action image(s) and corresponding current image(s) can be processed, using a trained critic network, to generate a value that indicates a probability of success of the robotic task if component(s) of the robot are traversed to the particular pose. When the value satisfies one or more conditions (e.g., satisfies a threshold), the robot can be controlled to cause the component(s) to traverse to the particular pose in performing the robotic task.

Abstract: A robotic gripping device is provided. The robotic gripping device includes a palm and a plurality of digits coupled to the palm. The robotic gripping device also includes a time-of-flight sensor arranged on the palm such that the time-of-flight sensor is configured to generate time-of-flight distance data in a direction between the plurality of digits. The robotic gripping device additionally includes an infrared camera, including an infrared illumination source, where the infrared camera is arranged on the palm such that the infrared camera is configured to generate grayscale image data in the direction between the plurality of digits.

Abstract: A robotic gripping device is provided. The robotic gripping device includes a palm and a plurality of digits coupled to the palm. The robotic gripping device also includes a time-of-flight sensor arranged on the palm such that the time-of-flight sensor is configured to generate time-of-flight distance data in a direction between the plurality of digits. The robotic gripping device additionally includes an infrared camera, including an infrared illumination source, where the infrared camera is arranged on the palm such that the infrared camera is configured to generate grayscale image data in the direction between the plurality of digits.

Abstract: Cascading variational autoencoder (“VAE”) models can be used to detect robot collisions while a robot is performing a task. For a current state of the robot, various implementations include a VAE used to generate a latent space of the current state, and a predictor network used to generate a predicted latent space for the current state. A collision can be determined based on a difference between the latent space for the current state and the predicted latent space for the current state.

Abstract: Methods, apparatus, and computer readable media applicable to balancing robots. Some implementations are directed to maintaining a given end effector pose (relative to a world frame) of an end effector of a balancing robot when there is a disturbance to a balancing base of the balancing robot. Some implementations are additionally or alternatively directed to transitioning a balancing robot from a fallen configuration to a balanced configuration. Some implementations are additionally or alternatively directed to mitigating the risk that a balancing robot will fall when interacting with actuable environmental objects (e.g., doors) and/or to lessen the disturbance to a balancing base when interacting with actuable environmental objects.

Abstract: Cascading variational autoencoder (“VAE”) models can be used to detect robot collisions while a robot is performing a task. For a current state of the robot, various implementations include a VAE used to generate a latent space of the current state, and a predictor network used to generate a predicted latent space for the current state. A collision can be determined based on a difference between the latent space for the current state and the predicted latent space for the current state.

Abstract: In one embodiment, a method includes receiving, from a first sensor on a robot, first sensor data indicative of an environment of the robot. The method also includes identifying, based on the first sensor data, an object of an object type in the environment of the robot, where the object type is associated with a classifier that takes sensor data from a predetermined pose relative to the object as input. The method further includes causing the robot to position a second sensor on the robot at the predetermined pose relative to the object. The method additionally includes receiving, from the second sensor, second sensor data indicative of the object while the second sensor is positioned at the predetermined pose relative to the object. The method further includes determining, by inputting the second sensor data into the classifier, a property of the object.

Abstract: Methods, apparatus, and computer readable media applicable to robots, such as balancing robots. Some implementations are directed to determining multiple measures of a property of a robot for a given time and determining a final measure of the property of the robot for the given time based on the multiple measures. One or more control commands may be generated based on the final measure of the property and provided to one or more actuators of the robot.

Abstract: In one embodiment, a method includes receiving, from a first sensor on a robot, first sensor data indicative of an environment of the robot. The method also includes identifying, based on the first sensor data, an object of an object type in the environment of the robot, where the object type is associated with a classifier that takes sensor data from a predetermined pose relative to the object as input. The method further includes causing the robot to position a second sensor on the robot at the predetermined pose relative to the object. The method additionally includes receiving, from the second sensor, second sensor data indicative of the object while the second sensor is positioned at the predetermined pose relative to the object. The method further includes determining, by inputting the second sensor data into the classifier, a property of the object.

Abstract: Generating and utilizing action image(s) that represent a candidate pose (e.g., a candidate end effector pose), in determining whether to utilize the candidate pose in performance of a robotic task. The action image(s) and corresponding current image(s) can be processed, using a trained critic network, to generate a value that indicates a probability of success of the robotic task if component(s) of the robot are traversed to the particular pose. When the value satisfies one or more conditions (e.g., satisfies a threshold), the robot can be controlled to cause the component(s) to traverse to the particular pose in performing the robotic task.

Abstract: A method is provided that includes controlling a robotic gripping device to cause a plurality of digits of the robotic gripping device to move towards each other in an attempt to grasp an object. The method also includes receiving, from at least one non-contact sensor on the robotic gripping device, first sensor data indicative of a region between the plurality of digits of the robotic gripping device. The method further includes receiving, from the at least one non-contact sensor on the robotic gripping device, second sensor data indicative of the region between the plurality of digits of the robotic gripping device, where the second sensor data is based on a different sensing modality than the first sensor data. The method additionally includes determining, using an object-in-hand classifier that takes as input the first sensor data and the second sensor data, a result of the attempt to grasp the object.

Abstract: A method is provided that includes controlling a robotic gripping device to cause a plurality of digits of the robotic gripping device to move towards each other in an attempt to grasp an object. The method also includes receiving, from at least one non-contact sensor on the robotic gripping device, first sensor data indicative of a region between the plurality of digits of the robotic gripping device. The method further includes receiving, from the at least one non-contact sensor on the robotic gripping device, second sensor data indicative of the region between the plurality of digits of the robotic gripping device, where the second sensor data is based on a different sensing modality than the first sensor data. The method additionally includes determining, using an object-in-hand classifier that takes as input the first sensor data and the second sensor data, a result of the attempt to grasp the object.

Abstract: A robotic gripping device is provided. The robotic gripping device includes a palm and a plurality of digits coupled to the palm. The robotic gripping device also includes a time-of-flight sensor arranged on the palm such that the time-of-flight sensor is configured to generate time-of-flight distance data in a direction between the plurality of digits. The robotic gripping device additionally includes an infrared camera, including an infrared illumination source, where the infrared camera is arranged on the palm such that the infrared camera is configured to generate grayscale image data in the direction between the plurality of digits.

Abstract: A robotic gripping device is provided. The robotic gripping device includes a palm and a plurality of digits coupled to the palm. The robotic gripping device also includes a time-of-flight sensor arranged on the palm such that the time-of-flight sensor is configured to generate time-of-flight distance data in a direction between the plurality of digits. The robotic gripping device additionally includes an infrared camera, including an infrared illumination source, where the infrared camera is arranged on the palm such that the infrared camera is configured to generate grayscale image data in the direction between the plurality of digits.