Abstract: A multicamera image processing system is disclosed. In various embodiments, image data is received from each of a plurality of sensors associated with a workspace, the image data comprising for each sensor in the plurality of sensors one or both of visual image information and depth information. Image data from the plurality of sensors is merged to generate a merged point cloud data. Segmentation is performed based on visual image data from at least a subset of the sensors in the plurality of sensors to generate a segmentation result. One or both of the merged point cloud data and the segmentation result is/are used to generate a merged three dimensional and segmented view of the workspace.

Abstract: A robotic singulation system is disclosed. In various embodiments, sensor data including image data associated with a workspace is received. The sensor data is used to generate a three dimensional view of at least a portion of the workspace, the three dimensional view including boundaries of a plurality of items present in the workspace. The three dimensional view as generated at successive points in time is used to model a flow of at least a subset of said plurality of items through at least a portion of the workspace. The model is used to determine and implement a plan to autonomously operate a robotic structure to pick one or more items from the workplace and place each item singly in a corresponding location in a singulation conveyance structure.

Abstract: A robot configured to use a gripper to grasp one or more tools is disclosed. In various embodiments, the robot comprises a robotic arm having a gripper disposed at a free moving end of the robotic arm, and a set of two or more tools configured to grasped or otherwise engaged by the gripper. Each tool in the set of two or more tools may be disposed in a corresponding tool holder, optionally attached to the robot or situated near the robot. The robot is configured to use the gripper to retrieve a selected tool from its tool holder to perform a task; use the tool to perform the task; and return the tool to its tool holder.

Abstract: A velocity control-based robotic system is disclosed. In various embodiments, sensor data is received from one or more sensors deployed in a physical space in which a robot is located. A processor is used to determine based at least in part on the sensor data an at least partly velocity-based trajectory along which to move an element comprising the robot. A command to implement the velocity-based trajectory is sent to the robot.

Abstract: A robotic system to control multiple robots to perform a task cooperatively is disclosed. A first robot determines to perform a task cooperatively with a second robot, moves independently to a first grasp position to grasp an object associated with the task, receives an indication that the second robot is prepared to perform the task cooperatively, and moves the object independently of the second robot in a leader mode along a trajectory determined by the first robot. The second robot assists the first robot in performing the task cooperatively, at least in part by moving independently to a second grasp position, grasping the object, and cooperating with the first robot to move the object, at least in part by operating in a follower mode of operation to maintain engagement with the object as the first robot moves the object along the trajectory.

Abstract: A robotic singulation system is disclosed. In various embodiments, sensor data including image data associated with a workspace is received. The sensor data is used to generate a three dimensional view of at least a portion of the workspace, the three dimensional view including boundaries of a plurality of items present in the workspace. The three dimensional view as generated at successive points in time is used to model a flow of at least a subset of said plurality of items through at least a portion of the workspace. The model is used to determine and implement a plan to autonomously operate a robotic structure to pick one or more items from the workplace and place each item singly in a corresponding location in a singulation conveyance structure.

Abstract: A robot configured to use a gripper to grasp one or more tools is disclosed. In various embodiments, the robot comprises a robotic arm having a gripper disposed at a free moving end of the robotic arm, and a set of two or more tools configured to grasped or otherwise engaged by the gripper. Each tool in the set of two or more tools may be disposed in a corresponding tool holder, optionally attached to the robot or situated near the robot. The robot is configured to use the gripper to retrieve a selected tool from its tool holder to perform a task; use the tool to perform the task; and return the tool to its tool holder.

Abstract: Data associated with a plurality of items to be stacked on or in a destination location is received. The data associated with the plurality of items enables a first attribute associated with a first item of the plurality of items to be identified. A plan to stack the items on or in the destination location is generated based at least in part on the received data. The plan includes with respect to the first item a plan, determined based at least in part on the first attribute, to place the first item in a stackable container along with one or more other items and to place the stackable container, once filled, in a corresponding location on or in the destination location. The plan is implemented at least in part by controlling a robotic arm to pick up the items and stack them on or in the destination location according to the plan.

Abstract: A robot is operated in an autonomous mode of operation to perform a plurality of tasks. It is determined that a later task of the plurality of tasks needs human assistance while performing a current task of the plurality of tasks. The human assistance is scheduled for the later task. A teleoperator is communicated with to perform the human assistance associated with the later task.

Abstract: A velocity control-based robotic system is disclosed. In various embodiments, sensor data is received from one or more sensors deployed in a physical space in which a robot is located. A processor is used to determine based at least in part on the sensor data an at least partly velocity-based trajectory along which to move an element comprising the robot. A command to implement the velocity-based trajectory is sent to the robot.

Abstract: A robotic system is disclosed. In various embodiments, sensor data is received. A plan is determined, based at least in part on the received sensor data, to use a robotic arm to build a stack of items comprising a plurality of items, the plan including with respect to at least a subset of the items a plan to move the item to an initial position on the stack and then reposition and use the robotic arm to pack the item more snugly against an adjacent surface. The plan is implemented at least in part by sending one or more commands to a robotic arm.

Abstract: A robotic system includes a robotic arm that includes a first joint that connects a first segment to a second segment, the first segment being connected at an end of the first segment opposite the first joint to a shoulder joint of the robotic arm and the second segment being connected at an end of the second segment opposite the first joint to an elbow joint of the robotic arm; and a processor coupled to the robotic arm and configured to receive an end effector trajectory and determine a motion plan to move the end effector through the end effector trajectory, including by using the first joint to vary the distance between the shoulder joint and the elbow joint, as and if needed, to realize the end effector trajectory while using the joints and links other than the first joint in a preferred pose.

Abstract: A robot is operated in an autonomous mode of operation in which the robot autonomously selects a strategy to pick up an item. The item is moved from an initial location to a destination location using the selected strategy. It is determined whether one or more strategies are available to pick up the item in response to the item being dropped while the item is being moved from the initial location to the destination location. It is determined that a further strategy is not available to pick up the item. In response to the determination that the further strategy is not available, a human intervention mode of operation is entered.

Abstract: A robotic system receives from a first agent included in a plurality of robotically controlled agents a request to be provided a motion plan to perform a first pick and place task assigned to the first agent. A first motion plan is determined for the first agent, including by taking into consideration a second motion plan associated with a second agent included in the plurality of robotically controlled agents to perform a second pick and place task assigned to the second agent, at least in part by considering a swept volume associated with at least a remaining uncompleted portion of the second motion plan as an obstacle with which the first agent will not collide while implementing the first motion plan.

Abstract: Techniques are disclosed to use robotic system simulation to control a robotic system. In various embodiments, a communication indicating an action to be performed by a robotic element is received from a robotic control system. Performance of the action by the robotic element is simulated. A state tracking data is updated to reflect a virtual change to one or more state variables as a result of simulated performance of the action. Successful completion of the action by the robotic element is reported to the robotic control system.

Abstract: Techniques are disclosed to use a robotic arm to palletize or depalletize diverse items. In various embodiments, data associated with a plurality of items to be stacked on or in a destination location is received. A plan to stack the items on or in the destination location is generated based at least in part on the received data. The plan is implemented at least in part by controlling a robotic arm of the robot to pick up the items and stack them on or in the receptacle according to the plan, including by for each item: using one or more first order sensors to move the item to a first approximation of a destination position for that item at the destination location; and using one or more second order sensors to snug the item into a final position.

Abstract: A variable payload robot is disclosed. In various embodiments, a robot includes two or more joints, each actuated by an associated joint motor and each joint motor having a different capacity, the robot comprising an end effector configured to grasp an object. A processor coupled to the robot is configured to determine based at least in part on the respective capacities of at least a subset of the joint motors and a payload related attribute of the object a plan and trajectory to move the object from a source location to a destination location.

Abstract: A value associated with a plurality of measurement objects that are embedded in a material associated with an intermediate layer of a plurality of layers associated with a tactile sensing unit is sensed. The value associated with the plurality of measurement objects is sensed by a sensor that is included a layer below the intermediate layer of the tactile sensing unit. An output of the sensor is used to make a determination associated with engagement of a robotic arm end effector with an item.

Abstract: A robotic singulation system is disclosed. In various embodiments, sensor data including image data associated with a workspace is received. The sensor data is used to generate a three dimensional view of at least a portion of the workspace, the three dimensional view including boundaries of a plurality of items present in the workspace. A grasp strategy is determined for each of at least a subset of items, and for each grasp strategy a corresponding probability of grasp success is computed. The grasp strategies and corresponding probabilities of grasp success are used to determine and implement a plan to autonomously operate a robotic structure to pick one or more items from the workplace and place each item singly in a corresponding location in a singulation conveyance structure.

Abstract: A robotic sorting system, method, and device is disclosed. The robotic sorting system includes a communication interface, and one or more processors coupled to the communication interface. The one or more processors are configured to (a) obtain item data via the communication interface, the item data including an indication of one or more items to be routed across a plurality of outbound stations, (b) obtain capability data for the plurality of outbound stations, the capability data indicating one or more capabilities for at least one outbound station, (c) determine a plan to route a selected item to a destination outbound station selected from among the plurality of outbound station, the plan being determined based at least in part on the item data and the capability data for the destination outbound station, and (d) cause the plan to be implemented to route the item to a particular handling path associated with the destination outbound station.