Abstract: In various embodiments communications latency and/or bandwidth of a communications connection between a device being controlled a remote operator workstation being used to control the device is measured. One or more parameters of the system, e.g., operator control stations and/or the device, e.g., robotic device, being remotely controlled, e.g., teleoperated, are altered in response to one or more of: i) communications latency, ii) communications bandwidth, iii) a task to be performed; and/or iv) environmental conditions. By changing such parameters, things such as maximum speed of device operation, a maximum acceleration or a maximum rate of movement of a device element such as forks of a forklift the device can be controlled or limited.

Abstract: A method for training an object detection system includes estimating a location of a first object in an environment based on a density cluster map generated from a plurality of images of the environment. The method also includes generating one or more negative training samples of the first object in the environment based on the plurality of images, each of the one or more negative training samples corresponding to a second object at a location in the environment that is different than the estimated location of the first object. The method further includes generating positive training samples from a set of images of the first object. The method also includes training the object detection system to detect the first object based on the positive training samples and the negative training sample.

Abstract: A method for generating positive and negative training samples is presented. The method includes identifying false positive images of an object based on multiple images of an environment. The method also includes generating positive training samples from a set of images of the object. The method further includes generating a negative training sample from the false positive image. The method still further includes training an object detection system based on the positive training samples and the negative training sample.

Abstract: A method includes providing a virtual representation of an environment of a robot, the virtual representation including an object representation of an object in the environment. The method further includes receiving manipulation input from a user to teleoperate the robot for manipulation of the object. The method also includes alerting the user to an alignment dimension based upon the manipulation input, receiving confirmation input from the user to engage the alignment dimension, and constraining at least one dimension of movement of the object according to the alignment dimension.

Abstract: A teleoperation system includes a robot comprising an actuator configured to move at least a portion of the robot, and a remote computing device comprising: one or more processors, one or more sensors communicatively coupled to the one or more processors, a non-transitory memory component communicatively coupled to the one or more processors, and machine readable instructions stored in the non-transitory memory component. The remote computing device obtains information about a user proximate to the remote computing device, identifies the user based on the obtained information, obtains an action of a user, retrieves an individual profile for the user based on the identified user, determines an intended instruction related to a task based on the action of the user related to the task and the individual profile for the user, and instructs the robot to implement the task with the actuator based on the intended instruction.

Abstract: A method includes providing a virtual representation of an environment of a robot. The virtual representation includes an object representation of an object in the environment. The method further includes determining an attribute of the object within the environment of the robot. The attribute includes at least one of occupancy data, force data, and deformation data pertaining to the object. The method further includes receiving a user command to control the robot to move with respect to the object, and modifying a received user command pertaining to the representation of the object based upon the attribute.

Abstract: A method for generating positive and negative training samples is presented. The method includes identifying false positive images of an object based on multiple images of an environment. The method also includes generating positive training samples from a set of images of the object. The method further includes generating a negative training sample from the false positive image. The method still further includes training an object detection system based on the positive training samples and the negative training sample.

Abstract: A method includes detecting an object in a real environment of a robot. The method further includes inferring an expected property of the object based upon a representation of the object within a representation of the real environment of the robot. The method also includes sensing, via a sensor of the robot, a presently-detected property of the object in the real environment corresponding to the expected property. The method still further includes detecting a conflict between the expected property of the object and the presently-detected property of the object.

Abstract: A method includes presenting a virtual representation of an environment of a robot, receiving a first user command to control the robot within the environment, rendering a predicted version of the virtual representation during a period of latency in which current data pertaining to the environment of the robot is not available, updating the predicted version of the virtual representation based upon a second user command received during the period of latency, and upon conclusion of the period of latency, reconciling the predicted version of the virtual representation with current data pertaining to the environment of the robot.

Abstract: A method includes presenting a virtual representation of an environment of a robot, receiving a first user command to control the robot within the environment, rendering a predicted version of the virtual representation during a period of latency in which current data pertaining to the environment of the robot is not available, updating the predicted version of the virtual representation based upon a second user command received during the period of latency, and upon conclusion of the period of latency, reconciling the predicted version of the virtual representation with current data pertaining to the environment of the robot.

Abstract: A method includes providing a virtual representation of an environment of a robot. The virtual representation includes an object representation of an object in the environment. The method further includes determining an attribute of the object within the environment of the robot. The attribute includes at least one of occupancy data, force data, and deformation data pertaining to the object. The method further includes receiving a user command to control the robot to move with respect to the object, and modifying a received user command pertaining to the representation of the object based upon the attribute.

Abstract: A method includes providing a virtual representation of an environment of a robot, the virtual representation including an object representation of an object in the environment. The method further includes receiving manipulation input from a user to teleoperate the robot for manipulation of the object. The method also includes alerting the user to an alignment dimension based upon the manipulation input, receiving confirmation input from the user to engage the alignment dimension, and constraining at least one dimension of movement of the object according to the alignment dimension.

Abstract: A method includes detecting an object in a real environment of a robot. The method further includes inferring an expected property of the object based upon a representation of the object within a representation of the real environment of the robot. The method also includes sensing, via a sensor of the robot, a presently-detected property of the object in the real environment corresponding to the expected property. The method still further includes detecting a conflict between the expected property of the object and the presently-detected property of the object.

Abstract: A teleoperation system includes a robot comprising an actuator configured to move at least a portion of the robot, and a remote computing device comprising: one or more processors, one or more sensors communicatively coupled to the one or more processors, a non-transitory memory component communicatively coupled to the one or more processors, and machine readable instructions stored in the non-transitory memory component. The remote computing device obtains information about a user proximate to the remote computing device, identifies the user based on the obtained information, obtains an action of a user, retrieves an individual profile for the user based on the identified user, determines an intended instruction related to a task based on the action of the user related to the task and the individual profile for the user, and instructs the robot to implement the task with the actuator based on the intended instruction.

Abstract: Systems, robots, and methods for generating three-dimensional skeleton representations of people are disclosed. A method includes generating, from a two-dimensional image, a two-dimensional skeleton representation of a person present in the two-dimensional image. The two-dimensional skeleton representation includes a plurality of joints and a plurality of links between individual joints of the plurality of joints. The method further includes positioning a cone around one or more links of the plurality of links, and identifying points of a depth cloud that intersect with the cone positioned around the one or more links of the two-dimensional skeleton. The points of the depth cloud are generated by a depth sensor and each point provides depth information. The method also includes projecting the two-dimensional skeleton representation into three-dimensional space using the depth information of the points that intersect with the cone, thereby generating the three-dimensional skeleton representation of the person.

Abstract: Systems, robots, and methods for generating three-dimensional skeleton representations of people are disclosed. A method includes generating, from a two-dimensional image, a two-dimensional skeleton representation of a person present in the two-dimensional image. The two-dimensional skeleton representation includes a plurality of joints and a plurality of links between individual joints of the plurality of joints. The method further includes positioning a cone around one or more links of the plurality of links, and identifying points of a depth cloud that intersect with the cone positioned around the one or more links of the two-dimensional skeleton. The points of the depth cloud are generated by a depth sensor and each point provides depth information. The method also includes projecting the two-dimensional skeleton representation into three-dimensional space using the depth information of the points that intersect with the cone, thereby generating the three-dimensional skeleton representation of the person.