Abstract: Data indicating a corresponding safety state information indicating an extent to which that operating zone currently is or is not available to the robotic system to perform tasks using one or more robotic instrumentalities associated with a robotic system is used, for each of a plurality of operating zones, to dynamically schedule and perform tasks associated with a higher level objective. The corresponding safety state information associated with the first operating zone indicating one of a plurality of safety states associated with a presence of a human worker in the first operating zone is received from one or more sensors associated with a first operating zone of the plurality of operating zones. Autonomous behavior of the one or more robotic instrumentalities is altered based on the one of the plurality of safety states associated with the presence of the human worker in the first operating zone indicated by the corresponding safety state information.

Abstract: A robotic singulation system is disclosed. In various embodiments, sensor data including image data associated with a workspace is received. The sensor data is used to generate a three dimensional view of at least a portion of the workspace, the three dimensional view including boundaries of a plurality of items present in the workspace. A grasp strategy is determined for each of at least a subset of items, and for each grasp strategy a corresponding probability of grasp success is computed. The grasp strategies and corresponding probabilities of grasp success are used to determine and implement a plan to autonomously operate a robotic structure to pick one or more items from the workplace and place each item singly in a corresponding location in a singulation conveyance structure.

Abstract: Techniques are disclosed to use robotic system simulation to control a robotic system. In various embodiments, a communication indicating an action to be performed by a robotic element is received from a robotic control system. Performance of the action by the robotic element is simulated. A state tracking data is updated to reflect a virtual change to one or more state variables as a result of simulated performance of the action. Successful completion of the action by the robotic element is reported to the robotic control system.

Abstract: A robotic line kitting system is disclosed. Data indicating for each of a plurality of operating zones comprising a workspace with which the robotic line kitting system is associated a corresponding safety state information is stored. The data is used to dynamically schedule and perform tasks associated with a higher level objective, including by operating the one or more robotic instrumentalities in one or more operating zones, if any, indicated by the data to currently be available to perform tasks using the one or more robotic instrumentalities and by not operating the one or more robotic instrumentalities in one or more operating zones, if any, indicated by the data to not currently be available to perform tasks using the one or more robotic instrumentalities.

Abstract: A robotic system to control multiple robots to perform a task cooperatively is disclosed. A first robot determines to perform a task cooperatively with a second robot, moves independently to a first grasp position to grasp an object associated with the task, receives an indication that the second robot is prepared to perform the task cooperatively, and moves the object independently of the second robot in a leader mode along a trajectory determined by the first robot. The second robot assists the first robot in performing the task cooperatively, at least in part by moving independently to a second grasp position, grasping the object, and cooperating with the first robot to move the object, at least in part by operating in a follower mode of operation to maintain engagement with the object as the first robot moves the object along the trajectory.

Abstract: A set of one or more potentially graspable features for one or more objects present in a workspace area are determined based on visual data received from a plurality of cameras. For each of at least a subset of the one or more potentially graspable features one or more corresponding grasp strategies are determined to grasp the feature with a robotic arm and end effector. A score associated with a probability of a successful grasp of a corresponding feature is determined with respect to each of a least a subset of said grasp strategies. A first feature of the one or more potentially graspable features is selected to be grasped using a selected grasp strategy based at least in part on a corresponding score associated with the selected grasp strategy with respect to the first feature. The robotic arm and the end effector are controlled to attempt to grasp the first feature using the selected grasp strategy.

Abstract: A robotic singulation system is disclosed. In various embodiments, sensor data including image data associated with a workspace is received. The sensor data is used to generate a three dimensional view of at least a portion of the workspace, the three dimensional view including boundaries of a plurality of items present in the workspace. The three dimensional view as generated at successive points in time is used to model a flow of at least a subset of said plurality of items through at least a portion of the workspace. The model is used to determine and implement a plan to autonomously operate a robotic structure to pick one or more items from the workplace and place each item singly in a corresponding location in a singulation conveyance structure.

Abstract: Techniques are disclosed to use a robotic arm to palletize or depalletize diverse items. In various embodiments, data associated with a plurality of items to be stacked on or in a destination location is received. A plan to stack the items on or in the destination location is generated based at least in part on the received data. The plan is implemented at least in part by controlling a robotic arm of the robot to pick up the items and stack them on or in the receptacle according to the plan, including by for each item: using one or more first order sensors to move the item to a first approximation of a destination position for that item at the destination location; and using one or more second order sensors to snug the item into a final position.

Abstract: Techniques are disclosed to perform robotic handling of soft products in non-rigid packaging. In various embodiments, sensor data associated with a workspace is received. An action to be performed in the workspace using one or more robotic elements is determined, the action including moving an end effector of one of the robotic elements relatively quickly to a location in proximity to an item to be grasped; actuating a grasping mechanism of the end effector to grasp the item using an amount of force and structures associated with minimized risk of damage to one or both of the item and its packaging; and using sensor data generated subsequent to the item being grasped to ensure the item has been grasped securely. Control communications are sent to the robotic element via the communication interface to cause robotic element to perform the action.

Abstract: A set of one or more potentially graspable features for one or more objects present in a workspace area are determined based on visual data received from a plurality of cameras. For each of at least a subset of the one or more potentially graspable features one or more corresponding grasp strategies are determined to grasp the feature with a robotic arm and end effector. A score associated with a probability of a successful grasp of a corresponding feature is determined with respect to each of a least a subset of said grasp strategies. A first feature of the one or more potentially graspable features is selected to be grasped using a selected grasp strategy based at least in part on a corresponding score associated with the selected grasp strategy with respect to the first feature. The robotic arm and the end effector are controlled to attempt to grasp the first feature using the selected grasp strategy.

Abstract: Techniques are disclosed to perform robotic handling of soft products in non-rigid packaging. In various embodiments, sensor data associated with a workspace is received. An action to be performed in the workspace using one or more robotic elements is determined, the action including moving an end effector of one of the robotic elements relatively quickly to a location in proximity to an item to be grasped; actuating a grasping mechanism of the end effector to grasp the item using an amount of force and structures associated with minimized risk of damage to one or both of the item and its packaging; and using sensor data generated subsequent to the item being grasped to ensure the item has been grasped securely. Control communications are sent to the robotic element via the communication interface to cause robotic element to perform the action.

Abstract: A kitting machine is disclosed. In various embodiments, the machine includes an actuator configured to move a surface or receptacle to a position associated with item retrieval; and a controller configured to control operation of the actuator to position an item in the position associated with item retrieval, in a manner that is synchronized at least in part with operation of a robotic retrieval device configured to retrieve the item from the position associated with item retrieval.

Abstract: A robotic singulation system is disclosed. In various embodiments, sensor data including image data associated with a workspace is received. The sensor data is used to generate a three dimensional view of at least a portion of the workspace, the three dimensional view including boundaries of a plurality of items present in the workspace. A grasp strategy is determined for each of at least a subset of items, and for each grasp strategy a corresponding probability of grasp success is computed. The grasp strategies and corresponding probabilities of grasp success are used to determine and implement a plan to autonomously operate a robotic structure to pick one or more items from the workplace and place each item singly in a corresponding location in a singulation conveyance structure.

Abstract: A robotic system to control multiple robots to perform a task cooperatively is disclosed. A first robot determines to perform a task cooperatively with a second robot, moves independently to a first grasp position to grasp an object associated with the task, receives an indication that the second robot is prepared to perform the task cooperatively, and moves the object independently of the second robot in a leader mode along a trajectory determined by the first robot. The second robot assists the first robot in performing the task cooperatively, at least in part by moving independently to a second grasp position, grasping the object, and cooperating with the first robot to move the object, at least in part by operating in a follower mode of operation to maintain engagement with the object as the first robot moves the object along the trajectory.

Abstract: A robotic kitting machine is disclosed. In various embodiments, a robotic arm is used to move an item to a location in proximity to a slot into which the item is to be inserted. Force information generated by a force sensor is received via a communication interface. The force sensor information is used to align a structure comprising the item with a corresponding cavity comprising the slot, and the item is inserted into the slot.

Abstract: A robotic singulation system is disclosed. In various embodiments, sensor data including image data associated with a workspace is received. The sensor data is used to generate a three dimensional view of at least a portion of the workspace, the three dimensional view including boundaries of a plurality of items present in the workspace. The three dimensional view as generated at successive points in time is used to model a flow of at least a subset of said plurality of items through at least a portion of the workspace. The model is used to determine and implement a plan to autonomously operate a robotic structure to pick one or more items from the workplace and place each item singly in a corresponding location in a singulation conveyance structure.

Abstract: The present application discloses a method, system, and computer system for sequencing a set of objects to be loaded into a transport container. The method includes (i) receiving an indication of the set of objects to be loaded into the transport container for transport from a source location to a destination location, (ii) determining, based at least in part on object information corresponding to the set of objects and resources available to load the set of objects, a sequence according to which the set of objects are to be loaded into the transport container, and (iii) providing the sequence to a robotic system to implement in connection with loading the set of objects to the transport container.

Abstract: An autonomous tray handling robotic system is disclosed. In various embodiments, data indicating a set of output stacks to be assembled is stored, each output stack including an associated set of trays each of a corresponding tray type. Operation of one or more robots is controlled, each robot being configured to grasp, move, and place one or more trays at a time, according to a plan, to iteratively pick trays from source stacks of trays and assemble the set of output stacks, including by building each output stack by successively placing on the output stack trays picked from one or more corresponding source stacks. Each of the robots comprises a robotic arm and a tray handling end effector configured to grasp, move, and place one or more trays without assistance from another robot.

Abstract: A multicamera image processing system is disclosed. In various embodiments, image data is received from each of a plurality of sensors associated with a workspace, the image data comprising for each sensor in the plurality of sensors one or both of visual image information and depth information. Image data from the plurality of sensors is merged to generate a merged point cloud data. Segmentation is performed based on visual image data from at least a subset of the sensors in the plurality of sensors to generate a segmentation result. One or both of the merged point cloud data and the segmentation result is/are used to generate a merged three dimensional and segmented view of the workspace.

Abstract: A kitting machine is disclosed. In various embodiments, the machine includes an actuator configured to move a surface or receptacle to a position associated with item retrieval; and a controller configured to control operation of the actuator to position an item in the position associated with item retrieval, in a manner that is synchronized at least in part with operation of a robotic retrieval device configured to retrieve the item from the position associated with item retrieval.