Abstract: Systems, methods, and other embodiments described herein relate to an interface for training a visuomotor policy to be device agnostic and controlling a robotic device using the visuomotor policy. In one embodiment, a method includes collecting sensor data about a robotic manipulator within an environment. The method includes pre-processing the sensor data to compensate for an observation latency of at least one sensor associated with the robotic manipulator. The method includes generating actions for the robotic manipulator to perform a task according to the sensor data and a visuomotor policy. The method includes controlling the robotic manipulator, including compensating for execution latency of the robotic manipulator in performing the actions.

Abstract: A method includes receiving an image of a scene, receiving a request to identify tasks that may be performed by a user utilizing a hand-held manipulation device, based on the image of the scene, in response to receiving the request to identify the tasks that may be performed by the user utilizing the hand-held manipulation device, analyzing the image of the scene to determine one or more suggested tasks that may be performed by the user utilizing the hand-held manipulation device, outputting the one or more suggested tasks that may be performed by the user utilizing the hand-held manipulation device, and receiving training data in response to outputting the one or more suggested tasks that may be performed by the user utilizing the hand-held manipulation device.

Abstract: Systems and methods for manipulating deformable objects using air are disclosed. In one embodiment, a system for manipulating a deformable object, the system includes a first robot arm having a first gripper, a second robot arm having a second gripper, a blower robot arm having an air pump, and one or more processors programmed to control the first robot arm and the second robot arm to grasp the deformable object using the first gripper and the second gripper, respectively, and to control the blower robot arm to perform a plurality of blowing actions onto the deformable object until an objective is satisfied.

Abstract: A method may include receiving an image of a robot, receiving a language instruction of a task to be performed by the robot, generating a plurality of image sequences of the robot performing the task based on the received image of the robot and the language instruction, selecting a first image sequence among the plurality of image sequences having a highest probability of performing the task, and determining a plurality of actions to be performed by a second robot to perform the task based on the first image sequence.

Abstract: A method includes receiving observation data comprising sensor data associated with a robot while one or more humans are controlling the robot to perform a specified task, receiving control data comprising control commands input by the one or more humans while controlling the robot to perform the specified task, adding Gaussian noise to the control data to generate noisy control data, and training a neural network, based on the noisy control data to receive the observation data and first Gaussian noise, and output an action sequence, comprising a plurality of action steps, to be performed by the robot to perform the specified task.

Abstract: Systems, methods, and other embodiments described herein relate to an interface for training a visuomotor policy to be device agnostic and controlling a robotic device using the visuomotor policy. In one embodiment, a method includes collecting sensor data about a robotic manipulator within an environment. The method includes pre-processing the sensor data to compensate for an observation latency of at least one sensor associated with the robotic manipulator. The method includes generating actions for the robotic manipulator to perform a task according to the sensor data and a visuomotor policy. The method includes controlling the robotic manipulator, including compensating for execution latency of the robotic manipulator in performing the actions.

Abstract: Systems and methods for manipulating deformable objects using air are disclosed. In one embodiment, a system for manipulating a deformable object, the system includes a first robot arm having a first gripper, a second robot arm having a second gripper, a blower robot arm having an air pump, and one or more processors programmed to control the first robot arm and the second robot arm to grasp the deformable object using the first gripper and the second gripper, respectively, and to control the blower robot arm to perform a plurality of blowing actions onto the deformable object until an objective is satisfied.