Abstract: A robotic singulation system is disclosed. In various embodiments, sensor data including image data associated with a workspace is received. The sensor data is used to generate a three dimensional view of at least a portion of the workspace, the three dimensional view including boundaries of a plurality of items present in the workspace. A grasp strategy is determined for each of at least a subset of items, and for each grasp strategy a corresponding probability of grasp success is computed. The grasp strategies and corresponding probabilities of grasp success are used to determine and implement a plan to autonomously operate a robotic structure to pick one or more items from the workplace and place each item singly in a corresponding location in a singulation conveyance structure.

Abstract: Techniques are disclosed to use robotic system simulation to control a robotic system. In various embodiments, a communication indicating an action to be performed by a robotic element is received from a robotic control system. Performance of the action by the robotic element is simulated. A state tracking data is updated to reflect a virtual change to one or more state variables as a result of simulated performance of the action. Successful completion of the action by the robotic element is reported to the robotic control system.

Abstract: A set of one or more potentially graspable features for one or more objects present in a workspace area are determined based on visual data received from a plurality of cameras. For each of at least a subset of the one or more potentially graspable features one or more corresponding grasp strategies are determined to grasp the feature with a robotic arm and end effector. A score associated with a probability of a successful grasp of a corresponding feature is determined with respect to each of a least a subset of said grasp strategies. A first feature of the one or more potentially graspable features is selected to be grasped using a selected grasp strategy based at least in part on a corresponding score associated with the selected grasp strategy with respect to the first feature. The robotic arm and the end effector are controlled to attempt to grasp the first feature using the selected grasp strategy.

Abstract: A robotic singulation system is disclosed. In various embodiments, sensor data including image data associated with a workspace is received. The sensor data is used to generate a three dimensional view of at least a portion of the workspace, the three dimensional view including boundaries of a plurality of items present in the workspace. The three dimensional view as generated at successive points in time is used to model a flow of at least a subset of said plurality of items through at least a portion of the workspace. The model is used to determine and implement a plan to autonomously operate a robotic structure to pick one or more items from the workplace and place each item singly in a corresponding location in a singulation conveyance structure.

Abstract: A set of one or more potentially graspable features for one or more objects present in a workspace area are determined based on visual data received from a plurality of cameras. For each of at least a subset of the one or more potentially graspable features one or more corresponding grasp strategies are determined to grasp the feature with a robotic arm and end effector. A score associated with a probability of a successful grasp of a corresponding feature is determined with respect to each of a least a subset of said grasp strategies. A first feature of the one or more potentially graspable features is selected to be grasped using a selected grasp strategy based at least in part on a corresponding score associated with the selected grasp strategy with respect to the first feature. The robotic arm and the end effector are controlled to attempt to grasp the first feature using the selected grasp strategy.

Abstract: A robotic singulation system is disclosed. In various embodiments, sensor data including image data associated with a workspace is received. The sensor data is used to generate a three dimensional view of at least a portion of the workspace, the three dimensional view including boundaries of a plurality of items present in the workspace. A grasp strategy is determined for each of at least a subset of items, and for each grasp strategy a corresponding probability of grasp success is computed. The grasp strategies and corresponding probabilities of grasp success are used to determine and implement a plan to autonomously operate a robotic structure to pick one or more items from the workplace and place each item singly in a corresponding location in a singulation conveyance structure.

Abstract: A robotic singulation system is disclosed. In various embodiments, sensor data including image data associated with a workspace is received. The sensor data is used to generate a three dimensional view of at least a portion of the workspace, the three dimensional view including boundaries of a plurality of items present in the workspace. The three dimensional view as generated at successive points in time is used to model a flow of at least a subset of said plurality of items through at least a portion of the workspace. The model is used to determine and implement a plan to autonomously operate a robotic structure to pick one or more items from the workplace and place each item singly in a corresponding location in a singulation conveyance structure.

Abstract: A multicamera image processing system is disclosed. In various embodiments, image data is received from each of a plurality of sensors associated with a workspace, the image data comprising for each sensor in the plurality of sensors one or both of visual image information and depth information. Image data from the plurality of sensors is merged to generate a merged point cloud data. Segmentation is performed based on visual image data from at least a subset of the sensors in the plurality of sensors to generate a segmentation result. One or both of the merged point cloud data and the segmentation result is/are used to generate a merged three dimensional and segmented view of the workspace.

Abstract: Techniques are disclosed to use a robotic arm to palletize or depalletize diverse items. In various embodiments, data associated with a plurality of items to be stacked on or in a destination location is received. A plan to stack the items on or in the destination location is generated based at least in part on the received data. The plan is implemented at least in part by controlling a robotic arm of the robot to pick up the items and stack them on or in the receptacle according to the plan, including by for each item: using one or more first order sensors to move the item to a first approximation of a destination position for that item at the destination location; and using one or more second order sensors to snug the item into a final position.

Abstract: A robotic system is disclosed. The system includes a memory configured to store estimated state information associated with a computer simulation of a robotic operation to stack a plurality of items on a pallet or other receptacle. The system includes one or more processors coupled to the communication interface and configured to perform the computer simulation. The computer simulation is performed at least in part by combining geometric model data based on idealized simulated robotic placement of each item with programmatically generated noise data. The programmatically generated noise data reflects an estimation of the effect that one or more sources of noise in a real-world physical workspace with which the computer simulation is associated would have on a real-world state of the plurality of items and/or the pallet or other receptacle if the plurality of items were stacked on the pallet or other receptacle as simulated in the computer simulation.

Abstract: A robotic system is disclosed. The system includes a memory that stores a machine learning-based model to provide a scoring function value for a candidate item placement on a pallet on which are plurality of items are to be stacked given a current state value of the pallet and a set of zero or more items placed previously. The system includes one or more processors that use the model to determine a corresponding score for each of a plurality of candidate placements for a next item to be placed and the current state value associated with the current state of the pallet and a set of zero or more items placed previously, select a selected placement based at least in part on the respective scores, control a robotic arm to place the next item according to the selected placement.

Abstract: A robotic system is disclosed. The system includes a memory that stores for each of a plurality of items a set of attribute values. The system includes a processor(s) that uses the attribute values to simulate the placement of items, including by determining, iteratively, for each next item a placement location at which to place the item on a simulated stack of items on the pallet, using the attribute values and a geometric model of where items have been simulated to have been placed to estimate a state of the stack after each of a subset of simulated placements, and using the estimated state to inform a next placement decision. The steps of determining for each next item a placement location and estimating the state of the stack until all of at least a subset of the plurality of items have been simulated as having been placed on the stack.

Abstract: A robotic system is disclosed. The system includes a communication interface that receives, from a sensor(s) deployed in a workspace, sensor data indicative of a current state of the workspace, the workspace comprising a pallet or other receptacle and a plurality of items stacked on or in the receptacle. The system includes one or more processors that control a robotic arm to place a first set of items on or in, or remove the first set of items from, the pallet or other receptacle, update a geometric model based on the first set of items placed on or in a receptacle, use the geometric model in combination with the sensor data to estimate a stack of one or more items on or in the receptacle, and use the estimated state to generate or update a plan to control the robotic arm to place a second set of items.

Abstract: A robotic system is disclosed. The system includes a communication interface configured to receive, from one or more sensors deployed in a workspace, sensor data indicative of a current state of the workspace, the workspace comprising a pallet or other receptacle and a plurality of items stacked on or in the receptacle. The system includes one or more processors that use a geometric model based at least in part on past item placements in combination with the sensor data to estimate a state of the pallet or other receptacle and one or more items stacked on or in the pallet or other receptacle, and use the estimated state to generate or update a plan to control a robotic arm to place a next item on or in, or remove a next item from, the pallet or other receptacle in a manner that avoids having the next item collide with any other item stacked on or in the pallet or other receptacle.

Abstract: A robotic system is disclosed. The system includes a communication interface that receives, from one or more sensors deployed in a workspace, sensor data indicative of a current state of the workspace. The system includes one or more processors that use the sensor data to estimate a state of one or both of the pallet or other receptacle and the set of zero or more items stacked on or in the receptacle, and use the estimated state to generate or update a plan to control a robotic arm to place a next set of items on or in, or remove the next set of items from, the pallet or other receptacle, the plan comprising an ordered sequence of item placements or removals. The plan is generated or updated based at least in part by performing a bounded tree search in which a subset of possible ordered sequences is explored.

Abstract: A robotic system is disclosed. The system includes a memory configured to store for each of a plurality of items a set of attribute values representing one or more physical attributes of the item. The system includes one or more processors coupled to the communication interface and configured to use the attribute values as inputs to a physic engine configured to compute the stability of a simulated stack of items comprising at least a subset of the plurality of items.

Abstract: Techniques are disclosed to use a robotic arm to palletize or depalletize diverse items. In various embodiments, data associated with a plurality of items to be stacked on or in a destination location is received. A plan to stack the items on or in the destination location is generated based at least in part on the received data. The plan is implemented at least in part by controlling a robotic arm of the robot to pick up the items and stack them on or in the receptacle according to the plan, including by for each item: using one or more first order sensors to move the item to a first approximation of a destination position for that item at the destination location; and using one or more second order sensors to snug the item into a final position.

Abstract: Techniques are disclosed to use robotic system simulation to control a robotic system. In various embodiments, a communication indicating an action to be performed by a robotic element is received from a robotic control system. Performance of the action by the robotic element is simulated. A state tracking data is updated to reflect a virtual change to one or more state variables as a result of simulated performance of the action. Successful completion of the action by the robotic element is reported to the robotic control system.

Abstract: A robotic singulation system is disclosed. In various embodiments, sensor data including image data associated with a workspace is received. The sensor data is used to generate a three dimensional view of at least a portion of the workspace, the three dimensional view including boundaries of a plurality of items present in the workspace. A grasp strategy is determined for each of at least a subset of items, and for each grasp strategy a corresponding probability of grasp success is computed. The grasp strategies and corresponding probabilities of grasp success are used to determine and implement a plan to autonomously operate a robotic structure to pick one or more items from the workplace and place each item singly in a corresponding location in a singulation conveyance structure.

Abstract: A robotic singulation system is disclosed. In various embodiments, sensor data including image data associated with a workspace is received. The sensor data is used to generate a three dimensional view of at least a portion of the workspace, the three dimensional view including boundaries of a plurality of items present in the workspace. The three dimensional view as generated at successive points in time is used to model a flow of at least a subset of said plurality of items through at least a portion of the workspace. The model is used to determine and implement a plan to autonomously operate a robotic structure to pick one or more items from the workplace and place each item singly in a corresponding location in a singulation conveyance structure.