Patents Assigned to MUJIN, Inc.
-
Patent number: 10953549Abstract: A method for operating a robotic system includes determining a discretized object model based on source sensor data; comparing the discretized object model to a packing plan or to master data; determining a discretized platform model based on destination sensor data; determining height measures based on the destination sensor data; comparing the discretized platform model and/or the height measures to an expected platform model and/or expected height measures; and determining one or more errors by (i) determining at least one source matching error by identifying one or more disparities between (a) the discretized object model and (b) the packing plan or the master data or (ii) determining at least one destination matching error by identifying one or more disparities between (a) the discretized platform model or the height measures and (b) the expected platform model or the expected height measures, respectively.Type: GrantFiled: February 26, 2020Date of Patent: March 23, 2021Assignee: MUJIN, Inc.Inventors: Rosen Nikolaev Diankov, Denys Kanunikov
-
Patent number: 10933527Abstract: A method for operating a robotic system including determining an initial pose of a target object based on imaging data; calculating a confidence measure associated with an accuracy of the initial pose; and determining that the confidence measure fails to satisfy a sufficiency condition; and deriving a motion plan accordingly for scanning an object identifier while transferring the target object from a start location to a task location.Type: GrantFiled: January 15, 2020Date of Patent: March 2, 2021Assignee: MUJIN, Inc.Inventor: Rosen Diankov
-
Patent number: 10906184Abstract: A computing system and a method for calibration verification is presented. The computing system is configured to perform a first calibration operation, and to control a robot arm to move a verification symbol to a reference location. The robot control system further receives, from a camera, a reference image of the verification symbol, and determines a reference image coordinate for the verification symbol. The robot control system further controls the robot arm to move the verification symbol to the reference location again during an idle period, receives an additional image of the verification symbol, and determines a verification image coordinate. The robot control system determines a deviation parameter value based the reference image coordinate and the verification image coordinate, and whether the deviation parameter value exceeds a defined threshold, and performs a second calibration operation if the threshold is exceeded.Type: GrantFiled: April 30, 2020Date of Patent: February 2, 2021Assignee: MUJIN, INC.Inventors: Russell Islam, Xutao Ye, Rosen Diankov
-
Patent number: 10870204Abstract: The present disclosure provides a control method of a robotic system that implements a high degree of cooperation between units including a robot and increases a storage efficiency of an operation object sufficiently. A control method includes: deriving an approach location at which the end effector grips an operation object; deriving a scan location for scanning an identifier of the operation object; and based on the approach location and the scan location, creating or deriving a control sequence to instruct the robot to execute the control sequence.Type: GrantFiled: January 24, 2020Date of Patent: December 22, 2020Assignee: MUJIN, Inc.Inventors: Rosen Nikolaev Diankov, Yoshiki Kanemoto, Denys Kanunikov
-
Patent number: 10773385Abstract: A method for operating a robotic system that includes calculating a base motion plan, wherein the base motion plan includes a sequence of commands or settings, or a combination thereof, that operates a robotic arm and a gripper to transfer a target object from a start location to a task location; receiving a contact measure while executing the base motion plan, wherein the contact measure represents an amount of grip of the gripper on the target object; and generating one or more actuator commands/settings that deviate from the base motion plan when the contact measure fails to satisfy a threshold, wherein the one or more actuator commands/settings thereof are configured to operate the robotic arm, the gripper, or a combination thereof to execute one or more response actions not included in the base motion plan.Type: GrantFiled: October 11, 2019Date of Patent: September 15, 2020Assignee: MUJIN, Inc.Inventors: Rosen Nikolaev Diankov, Huan Liu, Denys Kanunikov
-
Patent number: 10766141Abstract: The present disclosure relates to operation of a robotic system to transfer an object from a source to a destination. The robotic system may implement one or more motion plans or portions thereof to operate a picking robot to grip and lift the object and place a transfer tray under the lifted object. The robotic system may further implement the one or more motion plans or portions thereof to place the object on the transfer tray and laterally displace the transfer tray and the object thereon toward the destination. The robotic system may operate a placement mechanism to transfer the object from the transfer tray to the destination.Type: GrantFiled: March 19, 2020Date of Patent: September 8, 2020Assignee: MUJIN, Inc.Inventors: Rosen Nikolaev Diankov, Shintaro Matsuoka, Yoshiki Kanemoto, Hironori Mizoguchi
-
Patent number: 10703584Abstract: A method for operating a robotic system includes obtaining and processing first data representative of an object at a start location; obtaining additional data as the object is transferred to a task location, wherein the additional data includes information regarding one or more edges and/or one or more surfaces separate from portions of the object captured in the first data; and creating registration data representative of the object based on the additional data.Type: GrantFiled: August 13, 2019Date of Patent: July 7, 2020Assignee: MUJIN, Inc.Inventors: Rosen Nikolaev Diankov, Huan Liu, Xutao Ye, Jose Jeronimo Moreira Rodrigues, Yoshiki Kanemoto, Jinze Yu, Russell Islam
-
Patent number: 10696494Abstract: A robotic system for arranging packages at a destination according to a stacking sequence. The robotic system uses a storage area for temporarily storing packages that arrive out-of-sequence until they are next-in-sequence for placement at the destination. The robotic system processes an incoming package, determines if it is next-in-sequence for placement at the destination, and if it is, places the package at the destination. On the other hand, if it is not next-in-sequence for placement at the destination, it stores the package in the storage area. A package in the storage area is transferred to the destination when it is next-in-sequence for placement at the destination. By using the temporary storage for storing out of sequence packages, the robotic system eliminates the need for receiving the packages in a stacking sequence, which also eliminates the need for sequencing machines.Type: GrantFiled: May 31, 2019Date of Patent: June 30, 2020Assignee: MUJIN, Inc.Inventors: Rosen Nikolaev Diankov, Denys Kanunikov
-
Patent number: 10696493Abstract: A method for operating a robotic system includes determining package groupings for placing available packages on a platform; generating a two-dimensional (2D) placement plan based on discretized models representative of the available packages and the platform; generating a three-dimensional (3D) stacking plan based on the 2D placement plan; and implementing the 3D stacking plan for placing the available packages on the platform.Type: GrantFiled: May 31, 2019Date of Patent: June 30, 2020Assignee: MUJIN, Inc.Inventors: Rosen Nilolaev Diankov, Denys Kanunikov
-
Patent number: 10679379Abstract: A method for operating a robotic system includes determining a discretized object model representative of a target object; determining a discretized platform model representative of a task location; determining height measures based on real-time sensor data representative of the task location; and dynamically deriving a placement location based on (1) overlapping the discretized object model and the discretized platform model for stacking objects at the task location and (2) calculating a placement score associated with the overlapping based on the height measures.Type: GrantFiled: May 31, 2019Date of Patent: June 9, 2020Assignee: MUJIN, Inc.Inventors: Rosen Nikolaev Diankov, Denys Kanunikov
-
Patent number: 10647528Abstract: A robotic system for arranging packages at a destination in a specified arrangement. The robotic system processes incoming packages, stores the packages in a temporary storage area, executes a simulate function to generate or update a simulated stacking plan, determines the occurrence of a palletizing trigger, and places the packages on the pallet according to the simulated stacking plan upon determining the occurrence of the palletizing trigger. The palletizing trigger can be one of a time limit trigger, a uniform layer trigger, a storage capacity trigger, or receiving a placement initiation command.Type: GrantFiled: May 31, 2019Date of Patent: May 12, 2020Assignee: MUJIN, Inc.Inventors: Rosen Nikolaev Diankov, Denys Kanunikov
-
Patent number: 10636172Abstract: A robot control system and a method for automatic camera calibration is presented. The robot control system includes a control circuit configured to control a robot arm to move a calibration pattern to at least one location within a camera field of view, and to receive a calibration image from a camera. The control circuit determines a first estimate of a first intrinsic camera parameter based on the calibration image. After the first estimate of the first intrinsic camera parameter is determined, the control circuit determines a first estimate of a second intrinsic camera parameter based on the first estimate of the first intrinsic camera parameter. These estimates are used to determine an estimate of a transformation function that describes a relationship between a camera coordinate system and a world coordinate system. The control circuit controls placement of the robot arm based on the estimate of the transformation function.Type: GrantFiled: July 8, 2019Date of Patent: April 28, 2020Assignee: MUJIN, INC.Inventors: Russell Islam, Xutao Ye, Rosen Diankov
-
Patent number: 10628966Abstract: A robot control system and a method for automatic camera calibration is presented. The robot control system includes a control circuit configured to control a robot arm to move a calibration pattern to at least one location within a camera field of view, and to receive a calibration image from a camera. The control circuit determines a first estimate of a first intrinsic camera parameter based on the calibration image. After the first estimate of the first intrinsic camera parameter is determined, the control circuit determines a first estimate of a second intrinsic camera parameter based on the first estimate of the first intrinsic camera parameter. These estimates are used to determine an estimate of a transformation function that describes a relationship between a camera coordinate system and a world coordinate system. The control circuit controls placement of the robot arm based on the estimate of the transformation function.Type: GrantFiled: June 28, 2019Date of Patent: April 21, 2020Assignee: MUJIN, INC.Inventors: Russell Islam, Xutao Ye, Rosen Diankov
-
Patent number: 10618172Abstract: A method for operating a robotic system includes determining a discretized object model based on source sensor data; comparing the discretized object model to a packing plan or to master data; determining a discretized platform model based on destination sensor data; determining height measures based on the destination sensor data; comparing the discretized platform model and/or the height measures to an expected platform model and/or expected height measures; and determining one or more errors by (i) determining at least one source matching error by identifying one or more disparities between (a) the discretized object model and (b) the packing plan or the master data or (ii) determining at least one destination matching error by identifying one or more disparities between (a) the discretized platform model or the height measures and (b) the expected platform model or the expected height measures, respectively.Type: GrantFiled: May 31, 2019Date of Patent: April 14, 2020Assignee: MUJIN, Inc.Inventors: Rosen Nikolaev Diankov, Denys Kanunikov
-
Patent number: 10614340Abstract: Systems and methods for processing spatial structure data are provided. The system accesses spatial structure data, which describes object structure, and which has depth information indicative of a plurality of layers for the object structure. The system further extracts, from the spatial structure data, a portion of the spatial structure data representative of one layer of the plurality of layers. The system identifies, from the portion of the spatial structure data, a plurality of vertices that describe a contour of the layer. Additionally, the system identifies convex corners of the layer based on the plurality of vertices and performs object recognition according to the convex corners.Type: GrantFiled: September 23, 2019Date of Patent: April 7, 2020Assignee: MUJIN, INC.Inventors: Jinze Yu, Xutao Ye
-
Patent number: 10596701Abstract: A method for operating a robotic system including determining an initial pose of a target object based on imaging data, wherein the initial pose is for estimating a resting orientation of the target object in a pickup area; calculating a confidence measure associated with the initial pose, wherein the confidence measure is for representing a likelihood of the initial pose being correct; and calculating a motion plan according to the confidence measure, the motion plan for executing a task based on picking up the target object from a start location, transferring the target object to a task location, and scanning one or more object identifiers between the start location and the task location.Type: GrantFiled: August 20, 2019Date of Patent: March 24, 2020Assignee: MUJIN, Inc.Inventor: Rosen Nikolaev Diankov
-
Patent number: 10583560Abstract: A robotic system includes: a control unit configured to: receive an object set including one or more object entries, wherein: the object entries correspond to source objects of an object source, each of the object entries are described by one or more object entry properties; receive sensor information representing one or more detectable object properties for detectable source objects of an object source; calculate an object match probability between the detectable source objects and the object entries based on a property correlation between the detectable object properties of the detectable source objects and the object entry properties of the object entries; generate an object identity approximation for each of the detectable source objects based on a comparison between the object match probability for each of the detectable source objects corresponding to a particular instance of the object entries; select a target object from the detectable source objects; generate an object handling strategy, for implemenType: GrantFiled: April 3, 2019Date of Patent: March 10, 2020Assignee: Mujin, Inc.Inventors: Jose Jeronimo Moreira Rodrigues, Xutao Ye, Jinze Yu, Rosen Diankov
-
Patent number: 10576636Abstract: A robot control system and a method for updating camera calibration is presented. The method comprises the robot control system performing a first camera calibration to determine camera calibration information, and outputting a first movement command based on the camera calibration information for a robot operation. The method further comprises outputting, after the first camera calibration, a second movement command to move a calibration pattern within a camera field of view, receiving one or more calibration images, and adding the one or more calibration images to a captured image set. The method further comprises performing a second camera calibration based on calibration images in the captured image set to determine updated camera calibration information, determining whether a deviation between the camera calibration information and the updated camera calibration information exceeds a defined threshold, and outputting a notification signal if the deviation exceeds the defined threshold.Type: GrantFiled: April 12, 2019Date of Patent: March 3, 2020Assignee: MUJIN, INC.Inventors: Russell Islam, Xutao Ye, Rosen Diankov
-
Patent number: 10576631Abstract: A method for operating a robotic system including determining an initial pose of a target object based on imaging data, wherein the initial pose is for estimating a resting orientation of the target object in a pickup area; calculating a confidence measure associated with the initial pose, wherein the confidence measure is for representing a likelihood of the initial pose being correct; and calculating a motion plan according to the confidence measure, the motion plan for executing a task based on picking up the target object from a start location, transferring the target object to a task location, and scanning one or more object identifiers between the start location and the task location.Type: GrantFiled: August 20, 2019Date of Patent: March 3, 2020Assignee: MUJIN, Inc.Inventor: Rosen Nikolaev Diankov
-
Patent number: 10576630Abstract: A system and method of operation of a robotic system including: receiving a sensor reading associated with a target object; generating a base plan for performing a task on the target object, wherein generating the base plan includes determining a grip point and one or more grip patterns associated with the grip point for gripping the target object based on a location of the grip point relative to a designated area, a task location, and another target object; implementing the base plan for performing the task by operating an actuation unit and one or more suction grippers according to a grip pattern rank, to generate an established grip on the target object, wherein the established grip is at a grip pattern location associated with the grip patterns; measuring the established grip; comparing the established grip to a force threshold; and re-gripping the target object based on the established grip falling below the force threshold.Type: GrantFiled: May 31, 2019Date of Patent: March 3, 2020Assignee: Mujin, Inc.Inventors: Rosen Diankov, Denys Kanunikov