Patents Assigned to MUJIN, Inc.
  • Patent number: 10953549
    Abstract: A method for operating a robotic system includes determining a discretized object model based on source sensor data; comparing the discretized object model to a packing plan or to master data; determining a discretized platform model based on destination sensor data; determining height measures based on the destination sensor data; comparing the discretized platform model and/or the height measures to an expected platform model and/or expected height measures; and determining one or more errors by (i) determining at least one source matching error by identifying one or more disparities between (a) the discretized object model and (b) the packing plan or the master data or (ii) determining at least one destination matching error by identifying one or more disparities between (a) the discretized platform model or the height measures and (b) the expected platform model or the expected height measures, respectively.
    Type: Grant
    Filed: February 26, 2020
    Date of Patent: March 23, 2021
    Assignee: MUJIN, Inc.
    Inventors: Rosen Nikolaev Diankov, Denys Kanunikov
  • Patent number: 10933527
    Abstract: A method for operating a robotic system including determining an initial pose of a target object based on imaging data; calculating a confidence measure associated with an accuracy of the initial pose; and determining that the confidence measure fails to satisfy a sufficiency condition; and deriving a motion plan accordingly for scanning an object identifier while transferring the target object from a start location to a task location.
    Type: Grant
    Filed: January 15, 2020
    Date of Patent: March 2, 2021
    Assignee: MUJIN, Inc.
    Inventor: Rosen Diankov
  • Patent number: 10906184
    Abstract: A computing system and a method for calibration verification is presented. The computing system is configured to perform a first calibration operation, and to control a robot arm to move a verification symbol to a reference location. The robot control system further receives, from a camera, a reference image of the verification symbol, and determines a reference image coordinate for the verification symbol. The robot control system further controls the robot arm to move the verification symbol to the reference location again during an idle period, receives an additional image of the verification symbol, and determines a verification image coordinate. The robot control system determines a deviation parameter value based the reference image coordinate and the verification image coordinate, and whether the deviation parameter value exceeds a defined threshold, and performs a second calibration operation if the threshold is exceeded.
    Type: Grant
    Filed: April 30, 2020
    Date of Patent: February 2, 2021
    Assignee: MUJIN, INC.
    Inventors: Russell Islam, Xutao Ye, Rosen Diankov
  • Patent number: 10870204
    Abstract: The present disclosure provides a control method of a robotic system that implements a high degree of cooperation between units including a robot and increases a storage efficiency of an operation object sufficiently. A control method includes: deriving an approach location at which the end effector grips an operation object; deriving a scan location for scanning an identifier of the operation object; and based on the approach location and the scan location, creating or deriving a control sequence to instruct the robot to execute the control sequence.
    Type: Grant
    Filed: January 24, 2020
    Date of Patent: December 22, 2020
    Assignee: MUJIN, Inc.
    Inventors: Rosen Nikolaev Diankov, Yoshiki Kanemoto, Denys Kanunikov
  • Patent number: 10773385
    Abstract: A method for operating a robotic system that includes calculating a base motion plan, wherein the base motion plan includes a sequence of commands or settings, or a combination thereof, that operates a robotic arm and a gripper to transfer a target object from a start location to a task location; receiving a contact measure while executing the base motion plan, wherein the contact measure represents an amount of grip of the gripper on the target object; and generating one or more actuator commands/settings that deviate from the base motion plan when the contact measure fails to satisfy a threshold, wherein the one or more actuator commands/settings thereof are configured to operate the robotic arm, the gripper, or a combination thereof to execute one or more response actions not included in the base motion plan.
    Type: Grant
    Filed: October 11, 2019
    Date of Patent: September 15, 2020
    Assignee: MUJIN, Inc.
    Inventors: Rosen Nikolaev Diankov, Huan Liu, Denys Kanunikov
  • Patent number: 10766141
    Abstract: The present disclosure relates to operation of a robotic system to transfer an object from a source to a destination. The robotic system may implement one or more motion plans or portions thereof to operate a picking robot to grip and lift the object and place a transfer tray under the lifted object. The robotic system may further implement the one or more motion plans or portions thereof to place the object on the transfer tray and laterally displace the transfer tray and the object thereon toward the destination. The robotic system may operate a placement mechanism to transfer the object from the transfer tray to the destination.
    Type: Grant
    Filed: March 19, 2020
    Date of Patent: September 8, 2020
    Assignee: MUJIN, Inc.
    Inventors: Rosen Nikolaev Diankov, Shintaro Matsuoka, Yoshiki Kanemoto, Hironori Mizoguchi
  • Patent number: 10703584
    Abstract: A method for operating a robotic system includes obtaining and processing first data representative of an object at a start location; obtaining additional data as the object is transferred to a task location, wherein the additional data includes information regarding one or more edges and/or one or more surfaces separate from portions of the object captured in the first data; and creating registration data representative of the object based on the additional data.
    Type: Grant
    Filed: August 13, 2019
    Date of Patent: July 7, 2020
    Assignee: MUJIN, Inc.
    Inventors: Rosen Nikolaev Diankov, Huan Liu, Xutao Ye, Jose Jeronimo Moreira Rodrigues, Yoshiki Kanemoto, Jinze Yu, Russell Islam
  • Patent number: 10696494
    Abstract: A robotic system for arranging packages at a destination according to a stacking sequence. The robotic system uses a storage area for temporarily storing packages that arrive out-of-sequence until they are next-in-sequence for placement at the destination. The robotic system processes an incoming package, determines if it is next-in-sequence for placement at the destination, and if it is, places the package at the destination. On the other hand, if it is not next-in-sequence for placement at the destination, it stores the package in the storage area. A package in the storage area is transferred to the destination when it is next-in-sequence for placement at the destination. By using the temporary storage for storing out of sequence packages, the robotic system eliminates the need for receiving the packages in a stacking sequence, which also eliminates the need for sequencing machines.
    Type: Grant
    Filed: May 31, 2019
    Date of Patent: June 30, 2020
    Assignee: MUJIN, Inc.
    Inventors: Rosen Nikolaev Diankov, Denys Kanunikov
  • Patent number: 10696493
    Abstract: A method for operating a robotic system includes determining package groupings for placing available packages on a platform; generating a two-dimensional (2D) placement plan based on discretized models representative of the available packages and the platform; generating a three-dimensional (3D) stacking plan based on the 2D placement plan; and implementing the 3D stacking plan for placing the available packages on the platform.
    Type: Grant
    Filed: May 31, 2019
    Date of Patent: June 30, 2020
    Assignee: MUJIN, Inc.
    Inventors: Rosen Nilolaev Diankov, Denys Kanunikov
  • Patent number: 10679379
    Abstract: A method for operating a robotic system includes determining a discretized object model representative of a target object; determining a discretized platform model representative of a task location; determining height measures based on real-time sensor data representative of the task location; and dynamically deriving a placement location based on (1) overlapping the discretized object model and the discretized platform model for stacking objects at the task location and (2) calculating a placement score associated with the overlapping based on the height measures.
    Type: Grant
    Filed: May 31, 2019
    Date of Patent: June 9, 2020
    Assignee: MUJIN, Inc.
    Inventors: Rosen Nikolaev Diankov, Denys Kanunikov
  • Patent number: 10647528
    Abstract: A robotic system for arranging packages at a destination in a specified arrangement. The robotic system processes incoming packages, stores the packages in a temporary storage area, executes a simulate function to generate or update a simulated stacking plan, determines the occurrence of a palletizing trigger, and places the packages on the pallet according to the simulated stacking plan upon determining the occurrence of the palletizing trigger. The palletizing trigger can be one of a time limit trigger, a uniform layer trigger, a storage capacity trigger, or receiving a placement initiation command.
    Type: Grant
    Filed: May 31, 2019
    Date of Patent: May 12, 2020
    Assignee: MUJIN, Inc.
    Inventors: Rosen Nikolaev Diankov, Denys Kanunikov
  • Patent number: 10636172
    Abstract: A robot control system and a method for automatic camera calibration is presented. The robot control system includes a control circuit configured to control a robot arm to move a calibration pattern to at least one location within a camera field of view, and to receive a calibration image from a camera. The control circuit determines a first estimate of a first intrinsic camera parameter based on the calibration image. After the first estimate of the first intrinsic camera parameter is determined, the control circuit determines a first estimate of a second intrinsic camera parameter based on the first estimate of the first intrinsic camera parameter. These estimates are used to determine an estimate of a transformation function that describes a relationship between a camera coordinate system and a world coordinate system. The control circuit controls placement of the robot arm based on the estimate of the transformation function.
    Type: Grant
    Filed: July 8, 2019
    Date of Patent: April 28, 2020
    Assignee: MUJIN, INC.
    Inventors: Russell Islam, Xutao Ye, Rosen Diankov
  • Patent number: 10628966
    Abstract: A robot control system and a method for automatic camera calibration is presented. The robot control system includes a control circuit configured to control a robot arm to move a calibration pattern to at least one location within a camera field of view, and to receive a calibration image from a camera. The control circuit determines a first estimate of a first intrinsic camera parameter based on the calibration image. After the first estimate of the first intrinsic camera parameter is determined, the control circuit determines a first estimate of a second intrinsic camera parameter based on the first estimate of the first intrinsic camera parameter. These estimates are used to determine an estimate of a transformation function that describes a relationship between a camera coordinate system and a world coordinate system. The control circuit controls placement of the robot arm based on the estimate of the transformation function.
    Type: Grant
    Filed: June 28, 2019
    Date of Patent: April 21, 2020
    Assignee: MUJIN, INC.
    Inventors: Russell Islam, Xutao Ye, Rosen Diankov
  • Patent number: 10618172
    Abstract: A method for operating a robotic system includes determining a discretized object model based on source sensor data; comparing the discretized object model to a packing plan or to master data; determining a discretized platform model based on destination sensor data; determining height measures based on the destination sensor data; comparing the discretized platform model and/or the height measures to an expected platform model and/or expected height measures; and determining one or more errors by (i) determining at least one source matching error by identifying one or more disparities between (a) the discretized object model and (b) the packing plan or the master data or (ii) determining at least one destination matching error by identifying one or more disparities between (a) the discretized platform model or the height measures and (b) the expected platform model or the expected height measures, respectively.
    Type: Grant
    Filed: May 31, 2019
    Date of Patent: April 14, 2020
    Assignee: MUJIN, Inc.
    Inventors: Rosen Nikolaev Diankov, Denys Kanunikov
  • Patent number: 10614340
    Abstract: Systems and methods for processing spatial structure data are provided. The system accesses spatial structure data, which describes object structure, and which has depth information indicative of a plurality of layers for the object structure. The system further extracts, from the spatial structure data, a portion of the spatial structure data representative of one layer of the plurality of layers. The system identifies, from the portion of the spatial structure data, a plurality of vertices that describe a contour of the layer. Additionally, the system identifies convex corners of the layer based on the plurality of vertices and performs object recognition according to the convex corners.
    Type: Grant
    Filed: September 23, 2019
    Date of Patent: April 7, 2020
    Assignee: MUJIN, INC.
    Inventors: Jinze Yu, Xutao Ye
  • Patent number: 10596701
    Abstract: A method for operating a robotic system including determining an initial pose of a target object based on imaging data, wherein the initial pose is for estimating a resting orientation of the target object in a pickup area; calculating a confidence measure associated with the initial pose, wherein the confidence measure is for representing a likelihood of the initial pose being correct; and calculating a motion plan according to the confidence measure, the motion plan for executing a task based on picking up the target object from a start location, transferring the target object to a task location, and scanning one or more object identifiers between the start location and the task location.
    Type: Grant
    Filed: August 20, 2019
    Date of Patent: March 24, 2020
    Assignee: MUJIN, Inc.
    Inventor: Rosen Nikolaev Diankov
  • Patent number: 10583560
    Abstract: A robotic system includes: a control unit configured to: receive an object set including one or more object entries, wherein: the object entries correspond to source objects of an object source, each of the object entries are described by one or more object entry properties; receive sensor information representing one or more detectable object properties for detectable source objects of an object source; calculate an object match probability between the detectable source objects and the object entries based on a property correlation between the detectable object properties of the detectable source objects and the object entry properties of the object entries; generate an object identity approximation for each of the detectable source objects based on a comparison between the object match probability for each of the detectable source objects corresponding to a particular instance of the object entries; select a target object from the detectable source objects; generate an object handling strategy, for implemen
    Type: Grant
    Filed: April 3, 2019
    Date of Patent: March 10, 2020
    Assignee: Mujin, Inc.
    Inventors: Jose Jeronimo Moreira Rodrigues, Xutao Ye, Jinze Yu, Rosen Diankov
  • Patent number: 10576636
    Abstract: A robot control system and a method for updating camera calibration is presented. The method comprises the robot control system performing a first camera calibration to determine camera calibration information, and outputting a first movement command based on the camera calibration information for a robot operation. The method further comprises outputting, after the first camera calibration, a second movement command to move a calibration pattern within a camera field of view, receiving one or more calibration images, and adding the one or more calibration images to a captured image set. The method further comprises performing a second camera calibration based on calibration images in the captured image set to determine updated camera calibration information, determining whether a deviation between the camera calibration information and the updated camera calibration information exceeds a defined threshold, and outputting a notification signal if the deviation exceeds the defined threshold.
    Type: Grant
    Filed: April 12, 2019
    Date of Patent: March 3, 2020
    Assignee: MUJIN, INC.
    Inventors: Russell Islam, Xutao Ye, Rosen Diankov
  • Patent number: 10576631
    Abstract: A method for operating a robotic system including determining an initial pose of a target object based on imaging data, wherein the initial pose is for estimating a resting orientation of the target object in a pickup area; calculating a confidence measure associated with the initial pose, wherein the confidence measure is for representing a likelihood of the initial pose being correct; and calculating a motion plan according to the confidence measure, the motion plan for executing a task based on picking up the target object from a start location, transferring the target object to a task location, and scanning one or more object identifiers between the start location and the task location.
    Type: Grant
    Filed: August 20, 2019
    Date of Patent: March 3, 2020
    Assignee: MUJIN, Inc.
    Inventor: Rosen Nikolaev Diankov
  • Patent number: 10576630
    Abstract: A system and method of operation of a robotic system including: receiving a sensor reading associated with a target object; generating a base plan for performing a task on the target object, wherein generating the base plan includes determining a grip point and one or more grip patterns associated with the grip point for gripping the target object based on a location of the grip point relative to a designated area, a task location, and another target object; implementing the base plan for performing the task by operating an actuation unit and one or more suction grippers according to a grip pattern rank, to generate an established grip on the target object, wherein the established grip is at a grip pattern location associated with the grip patterns; measuring the established grip; comparing the established grip to a force threshold; and re-gripping the target object based on the established grip falling below the force threshold.
    Type: Grant
    Filed: May 31, 2019
    Date of Patent: March 3, 2020
    Assignee: Mujin, Inc.
    Inventors: Rosen Diankov, Denys Kanunikov