Abstract: A robot system controller and control method that implement sophisticated cooperation among units is provided. The controller may include: a data acquisition unit adapted to acquire first data including information about the handling object and the shelf before storing or retrieving the handling object in/from the shelf; a data storage unit; and a robot control unit adapted to select and transport the shelf to an access position before storing or retrieving the handling object in/from the shelf, create or acquire a control sequence for storing or retrieving the handling object in/from the shelf, instruct the transport robot to execute a task of transporting the shelf to the access position, and instruct the handling robot to execute a task of storing or retrieving the handling object in/from the shelf.

Abstract: A control apparatus for controlling operation of a work robot for performing work inside a target region using a manipulator includes a trajectory information acquiring unit for acquiring N?1 or N pieces of trajectory information respectively indicating N?1 or N trajectories connecting N work regions where the work robot performs a series of work operations in order of a series of work operations; a classifying unit for classifying the N?1 or N trajectories as (i) trajectories that need correction or (ii) trajectories that do not need correction; and a trajectory planning unit for planning a trajectory of a tip of the manipulator between two work regions relating to the each of the one or more trajectories, for each of the one or more trajectories classified as a trajectory that needs correction by the classifying unit.

Abstract: A control apparatus includes a first information acquiring section that acquires three-dimensional information of a first region of surfaces of a plurality of objects, the information being obtained by imaging or scanning the plurality of objects from a first location; a second information acquiring section that acquires three-dimensional information of a second region of surfaces of the plurality of objects, the information being obtained by imaging or scanning the plurality of objects from a second location; and a combining section that generates information indicating three-dimensional shapes of at least a portion of the surfaces of the plurality of objects, based on the three-dimensional information of the first region acquired by the first information acquiring section and the three-dimensional information of the second region acquired by the second information acquiring section.

Abstract: A method of operating a robotic system includes: receiving first image data representative of a package surface; identifying a pair of edges based on the first image data; determining a minimum viable region based on the pair of edges; receiving second image data representative of the package after the lift; and creating registration data based on the first and second image data.

Abstract: A robot control system and a method for automatic camera calibration is presented. The robot control system includes a control circuit configured to control a robot arm to move a calibration pattern to at least one location within a camera field of view, and to receive a calibration image from a camera. The control circuit determines a first estimate of a first intrinsic camera parameter based on the calibration image. After the first estimate of the first intrinsic camera parameter is determined, the control circuit determines a first estimate of a second intrinsic camera parameter based on the first estimate of the first intrinsic camera parameter. These estimates are used to determine an estimate of a transformation function that describes a relationship between a camera coordinate system and a world coordinate system. The control circuit controls placement of the robot arm based on the estimate of the transformation function.

Abstract: A robot control system and a method for automatic camera calibration is presented. The robot control system includes a control circuit configured to control a robot arm to move a calibration pattern to at least one location within a camera field of view, and to receive a calibration image from a camera. The control circuit determines a first estimate of a first intrinsic camera parameter based on the calibration image. After the first estimate of the first intrinsic camera parameter is determined, the control circuit determines a first estimate of a second intrinsic camera parameter based on the first estimate of the first intrinsic camera parameter. These estimates are used to determine an estimate of a transformation function that describes a relationship between a camera coordinate system and a world coordinate system. The control circuit controls placement of the robot arm based on the estimate of the transformation function.

Abstract: A robotic system includes: a control unit configured to: receive an object set including one or more object entries, wherein: the object entries correspond to source objects of an object source, each of the object entries are described by one or more object entry properties; receive sensor information representing one or more detectable object properties for detectable source objects of an object source; calculate an object match probability between the detectable source objects and the object entries based on a property correlation between the detectable object properties of the detectable source objects and the object entry properties of the object entries; generate an object identity approximation for each of the detectable source objects based on a comparison between the object match probability for each of the detectable source objects corresponding to a particular instance of the object entries; select a target object from the detectable source objects; generate an object handling strategy, for implemen

Abstract: A robot control system and a method for updating camera calibration is presented. The method comprises the robot control system performing a first camera calibration to determine camera calibration information, and outputting a first movement command based on the camera calibration information for a robot operation. The method further comprises outputting, after the first camera calibration, a second movement command to move a calibration pattern within a camera field of view, receiving one or more calibration images, and adding the one or more calibration images to a captured image set. The method further comprises performing a second camera calibration based on calibration images in the captured image set to determine updated camera calibration information, determining whether a deviation between the camera calibration information and the updated camera calibration information exceeds a defined threshold, and outputting a notification signal if the deviation exceeds the defined threshold.

Abstract: A system and method of operation of a robotic system including: receiving a sensor reading associated with a target object; generating a base plan for performing a task on the target object, wherein generating the base plan includes determining a grip point and one or more grip patterns associated with the grip point for gripping the target object based on a location of the grip point relative to a designated area, a task location, and another target object; implementing the base plan for performing the task by operating an actuation unit and one or more suction grippers according to a grip pattern rank, to generate an established grip on the target object, wherein the established grip is at a grip pattern location associated with the grip patterns; measuring the established grip; comparing the established grip to a force threshold; and re-gripping the target object based on the established grip falling below the force threshold.

Abstract: A method of operating a package handling system includes: receiving first image data representative of a first surface image of a package surface; identifying a pair of intersecting edges for the package surface based on the first image; determining a minimum viable region based on the pair of edges, the minimum viable region for gripping and lifting the package; receiving second image data representative of the package after the lift; and creating registration data based on the third image data.

Abstract: A method of operating a package handling system includes: receiving first image data representative of a first surface image of a package surface; identifying a pair of intersecting edges for the package surface based on the first image; determining a minimum viable region based on the pair of edges, the minimum viable region for gripping and lifting the package; receiving second image data representative of the package after the lift; and creating registration data based on the third image data.

Abstract: A robot control system and a method for camera calibration verification is presented. The robot control system is configured to perform a first camera calibration, and to control a robot arm to move a verification symbol to a reference location. The robot control system further receives, from a camera, a reference image of the verification symbol, and determines a reference image coordinate for the verification symbol. The robot control system further controls the robot arm to move the verification symbol to the reference location again during an idle period, receives an additional image of the verification symbol, and determines a verification image coordinate. The robot control system determines a deviation parameter value based the reference image coordinate and the verification image coordinate, and whether the deviation parameter value exceeds a defined threshold, and performs a second camera calibration if the threshold is exceeded.

Abstract: A trajectory information generating unit is included, the trajectory information generating unit generating trajectory information that defines a trajectory for which a picking hand picks one of one or more works stored in one supply container included in one or more supply containers and arranges the work in one arrangement container included in one or more arrangement containers. After performing a process of generating trajectory information representing a first work included in the one or more works and before a movement of the first work is completed, the trajectory information generating unit may generate trajectory information representing a second work included in the one or more work and different from the first work by using an operation result in the process of generating the trajectory information regarding the first work.

Abstract: A point cloud information acquiring unit for acquiring point cloud information of a region including at least one of the picking hand and the work at a time which is at least one of (i) when the picking hand picks the work at the first position, (ii) when the picking hand moves the work out of the first container, (iii) when the picking hand moves the work in the second container and (iv) when the picking hand arranges the work at the second position; and an abnormality detection unit for detecting an abnormality based on the point cloud information acquired by the point cloud information acquiring unit are included.

Abstract: Automatic calculation of a trajectory by taking the interference with an obstacle into account may be performed by calculating trajectory information representing a trajectory on which (i) the picking hand picks, on the first position, the work with a posture associated with the first position included in the first combination; and (ii) the picking hand arranges, on the second position, the work with a posture associated with the second position included in the first combination, and determining whether an interference is present or absent on the trajectory represented by the trajectory information calculated in the calculating a trajectory.

Abstract: A three-dimensional information acquiring unit for acquiring three-dimensional information of a target object, a gripping information acquiring unit for acquiring gripping information that defines a method of gripping the target object by using a picking hand, and an information management unit for storing, in a storage apparatus in association with each other, (i) three-dimensional information of the target object acquired by the three-dimensional information acquiring unit and (ii) gripping information acquired by the gripping information acquiring unit are included. The three-dimensional information of the target object may include at least one of origin position information and the coordinate axis information. A two-dimensional information acquiring unit for acquiring two-dimensional information of the target object may be further included.

Abstract: An trajectory information generating unit for generating trajectory information defining a trajectory for which a picking hand picks a work at a first position and arranges the work at a second position, an execution control unit for operating a picking apparatus based on trajectory information generated by the trajectory information generating unit, and an execution time estimating unit for estimating a period of time from when the picking apparatus receives an instruction for starting an operation on a work to a time when the operation of the picking apparatus on the work is ended are included. The trajectory information generating unit may adjust an amount of calculation based on an estimation result of the execution time estimating unit.

Abstract: A method for operating a robotic system that includes calculating a base motion plan, wherein the base motion plan includes a sequence of commands or settings, or a combination thereof, that operates a robotic arm and a gripper to transfer a target object from a start location to a task location; receiving a contact measure while executing the base motion plan, wherein the contact measure represents an amount of grip of the gripper on the target object; and generating one or more actuator commands/settings that deviate from the base motion plan when the contact measure fails to satisfy a threshold, wherein the one or more actuator commands/settings thereof are configured to operate the robotic arm, the gripper, or a combination thereof to execute one or more response actions not included in the base motion plan.

Abstract: A robot control system and a method for camera calibration verification is presented. The robot control system is configured to perform a first camera calibration, and to control a robot arm to move a verification symbol to a reference location. The robot control system further receives, from a camera, a reference image of the verification symbol, and determines a reference image coordinate for the verification symbol. The robot control system further controls the robot arm to move the verification symbol to the reference location again during an idle period, receives an additional image of the verification symbol, and determines a verification image coordinate. The robot control system determines a deviation parameter value based the reference image coordinate and the verification image coordinate, and whether the deviation parameter value exceeds a defined threshold, and performs a second camera calibration if the threshold is exceeded.

Abstract: A method of operating a package handling system includes: receiving first image data representative of a first surface image of a package surface; identifying a pair of intersecting edges for the package surface based on the first image; determining a minimum viable region based on the pair of edges, the minimum viable region for gripping and lifting the package; receiving second image data representative of the package after the lift; and creating registration data based on the third image data.