Patents Assigned to MUJIN, Inc.
  • Patent number: 12380671
    Abstract: A computing system including a processing circuit in communication with a camera having a field of view. The processing circuit is configured to perform operations related to detecting, identifying, and retrieving objects disposed amongst a plurality of objects. The processing circuit may be configured to perform operations related to object recognition template generation, feature generation, hypothesis generation, hypothesis refinement, and hypothesis validation.
    Type: Grant
    Filed: August 9, 2022
    Date of Patent: August 5, 2025
    Assignee: MUJIN, INC.
    Inventors: Ahmed Abouelela, Hamdi Sahloul, Jose Jeronimo Moreira Rodrigues, Xutao Ye, Jinze Yu
  • Patent number: 12367612
    Abstract: A robot control system and a method for automatic camera calibration is presented. The robot control system includes a control circuit configured to determine all corner locations of an imaginary cube that fits within a camera field of view, and determine a plurality of locations that are distributed on or throughout the imaginary cube. The control circuit is further configured to control a robot arm to move a calibration pattern to the plurality of locations, and to receive a plurality of calibration images corresponding to the plurality of locations, and to determine respective estimates of intrinsic camera parameters based on the plurality of calibration images, and to determine an estimate of a transformation function that describes a relationship between a camera coordinate system and a world coordinate system. The control circuit is further configured to control placement of the robot arm based on the estimate of the transformation function.
    Type: Grant
    Filed: July 16, 2019
    Date of Patent: July 22, 2025
    Assignee: MUJIN, INC.
    Inventors: Russell Islam, Rosen Diankov, Xutao Ye
  • Patent number: 12343871
    Abstract: An object gripping assembly and associated systems and methods are disclosed herein. In some embodiments, the object gripping assembly includes a first carrying plate with a first mounting track extending along a first axis and two or more second carrying plates movably carried by the first mounting track. Each of the two or more second carrying plates can include a second mounting track extending along a second axis at least and extendable gripping components movably carried by the second mounting track. A first pitch adjusting component can be operably coupled the two or more second carrying plates to controllably change the pitch of the two or more second carrying plates along the first mounting track. A second pitch adjusting component can be operably coupled to the extendable gripping components on a corresponding second carrying plate to controllably change the pitch of the extendable gripping components along the second mounting track.
    Type: Grant
    Filed: October 20, 2021
    Date of Patent: July 1, 2025
    Assignee: Mujin, Inc.
    Inventors: Lei Lei, Xu Chen, Yixuan Zhang, Pai Zheng, Yufan Du, Hironori Mizoguchi, Shekhar Gupta, Rosen Nikolaev Diankov
  • Patent number: 12319521
    Abstract: A computing system including a processing circuit in communication with a robot and a camera having a field of view. The processing circuit obtains image information based on the objects in the field of view and a loading environment, the loading environment which includes loading areas, an object queue, and a buffer zone. The computing system is configured to use the obtained image information in motion planning operations for the retrieval and placement of objects from the object queue into the loading environment. Pallets provided within the loading environment (i.e., within the loading areas) are dedicated to receiving objects having corresponding object type identifiers. The computer system further uses the image information to determine the fill status of pallets existing within the loading environment, and whether new pallets need to be brought into the loading environment and/or swapped out with existing pallets to account for future planning and placement operations.
    Type: Grant
    Filed: August 9, 2022
    Date of Patent: June 3, 2025
    Assignee: MUJIN, INC.
    Inventors: Denys Kanunikov, Ziyan Zhou, Xutao Ye, Rosen Nikolaev Diankov
  • Patent number: 12314901
    Abstract: A system and method for operating a robotic system to register unrecognized objects is disclosed. The robotic system may use first image data representative of an unrecognized object to derive an initial minimum viable region (MVR). The robotic system may analyze second image data representative of the unrecognized object to detect a condition representative of an accuracy of the initial MVR. The robotic system may register the initial MVR or an adjustment thereof based on the detected condition.
    Type: Grant
    Filed: September 15, 2023
    Date of Patent: May 27, 2025
    Assignee: MUJIN, Inc.
    Inventors: Rosen Nikolaev Diankov, Russell Islam, Xutao Ye
  • Patent number: 12311551
    Abstract: A controller for a robotic system is provided. The controller may derive first data including information about operation objects and information about pallet before loading and/or unloading of the operation object on/from the pallet. The controller may derive a control sequence for loading and/or unloading the operation object on/from the pallet based on the first data before execution of the corresponding loading/unloading task. Based on the control sequence, the controller may instruct a robot to execute the task of loading and/or unloading the operation object on/from the pallet.
    Type: Grant
    Filed: May 12, 2021
    Date of Patent: May 27, 2025
    Assignee: MUJIN, Inc.
    Inventors: Isamu Arase, Satoshi Tendo, Rosen Nikolaev Diankov
  • Patent number: 12290944
    Abstract: A system and method for estimating aspects of target objects and/or associated task implementations is disclosed.
    Type: Grant
    Filed: April 28, 2022
    Date of Patent: May 6, 2025
    Assignee: MUJIN, Inc.
    Inventors: Jinze Yu, Jose Jeronimo Moreira Rodrigues, Rosen Nikolaev Diankov
  • Patent number: 12280967
    Abstract: A system and method for operating a robotic system to use two or more tools to manipulate objects. The robotic system may coordinate planning processes and/or plan implementations based on grouping the objects.
    Type: Grant
    Filed: May 10, 2024
    Date of Patent: April 22, 2025
    Assignee: MUJIN, Inc.
    Inventors: Shintaro Matsuoka, Hironori Mizoguchi, Rosen Nikolaev Diankov
  • Patent number: 12283064
    Abstract: A control apparatus includes a first information acquiring section that acquires three-dimensional information of a first region of surfaces of a plurality of objects, the information being obtained by imaging or scanning the plurality of objects from a first location; a second information acquiring section that acquires three-dimensional information of a second region of surfaces of the plurality of objects, the information being obtained by imaging or scanning the plurality of objects from a second location; and a combining section that generates information indicating three-dimensional shapes of at least a portion of the surfaces of the plurality of objects, based on the three-dimensional information of the first region acquired by the first information acquiring section and the three-dimensional information of the second region acquired by the second information acquiring section.
    Type: Grant
    Filed: December 1, 2023
    Date of Patent: April 22, 2025
    Assignee: MUJIN, INC.
    Inventors: Rosen Diankov, Xutao Ye, Ziyan Zhou
  • Patent number: 12275551
    Abstract: A multi-purpose labeling system can include a conveyor, a visual analysis module, and a labeling assembly. The conveyor can move an object in a first direction. The visual analysis module can include an optical sensor directed toward the conveyor to generate image data depicting the object. The labeling assembly can be spaced from the conveyor in a second direction and include a printer, a labeling module, and an alignment assembly. The printer can print a label based on the image data, and the labeling module can have a labeling plate for receiving the label. The alignment assembly can include a lateral-motion module, a vertical-motion module, and a rotary module for moving the labeling module along or about the first, the second, and a third direction, and can place the labeling plate adjacent to an object surface.
    Type: Grant
    Filed: August 10, 2022
    Date of Patent: April 15, 2025
    Assignee: Mujin, Inc.
    Inventors: Lei Lei, Yixuan Zhang, Xu Chen, Yi Xu, Brandon Coats, Rosen Nikolaev Diankov
  • Patent number: 12272070
    Abstract: Systems and methods for processing spatial structure data are provided. The system accesses spatial structure data, which describes object structure, and which has depth information indicative of a plurality of layers for the object structure. The system further extracts, from the spatial structure data, a portion of the spatial structure data representative of one layer of the plurality of layers. The system identifies, from the portion of the spatial structure data, a plurality of vertices that describe a contour of the layer. Additionally, the system identifies convex corners of the layer based on the plurality of vertices and performs object recognition according to the convex corners.
    Type: Grant
    Filed: September 1, 2023
    Date of Patent: April 8, 2025
    Assignee: MUJIN, INC.
    Inventors: Jinze Yu, Xutao Ye
  • Patent number: 12269164
    Abstract: A method and computing system for transferring objects between a source repository and a destination repository is provided. The computing system is configured to operate by a combination of pre-planned and image base trajectories to improve speed and reliability of object transfer. The computing system is configured to capture image information of repositories and use the captured information to alter or adjust pre-planned trajectories to improve performance.
    Type: Grant
    Filed: April 29, 2022
    Date of Patent: April 8, 2025
    Assignee: MUJIN, INC.
    Inventors: Xutao Ye, Hamdi Sahloul, Puttichai Lertkultanon, Rosen Nikolaev Diankov
  • Patent number: 12263577
    Abstract: A control apparatus includes a force sensation information acquiring unit for acquiring force sensation information representing a magnitude of at least one of a force and a torque at a distal end of a manipulator while a target item is being transported; a mass information acquiring unit for acquiring mass information representing a mass of the item; a plan information acquiring unit for acquiring plan information representing content of a plan of a trajectory; a force sensation estimating unit for estimating a magnitude of at least one of a force and a torque to be detected at the distal end of the manipulator; and a first detecting unit for detecting an abnormality, based on the magnitude of at least one of the force and the torque represented by the force sensation information and the magnitude of at least one of the force and the torque estimated by the estimating unit.
    Type: Grant
    Filed: January 12, 2022
    Date of Patent: April 1, 2025
    Assignee: MUJIN, INC.
    Inventors: Yoshiki Kanemoto, Rosen Diankov
  • Patent number: 12263598
    Abstract: An trajectory information generating unit for generating trajectory information defining a trajectory for which a picking hand picks a work at a first position and arranges the work at a second position, an execution control unit for operating a picking apparatus based on trajectory information generated by the trajectory information generating unit, and an execution time estimating unit for estimating a period of time from when the picking apparatus receives an instruction for starting an operation on a work to a time when the operation of the picking apparatus on the work is ended are included. The trajectory information generating unit may adjust an amount of calculation based on an estimation result of the execution time estimating unit.
    Type: Grant
    Filed: April 19, 2023
    Date of Patent: April 1, 2025
    Assignee: MUJIN, INC.
    Inventor: Rosen Diankov
  • Patent number: 12246438
    Abstract: A tool system for a robotic arm is provided. The tool system includes a tool support structure; a tool structure, coupled to the tool support structure, having at least one structural member; a connector mechanically coupled to the structural member and shaped for engaging the tool support structure; and a control unit mechanically coupled to the at least one structural member and configured to sense and adjust performance of the tool structure based on determining a connection to the tool support structure.
    Type: Grant
    Filed: September 28, 2022
    Date of Patent: March 11, 2025
    Assignee: Mujin, Inc.
    Inventors: Lei Lei, Yixuan Zhang
  • Patent number: 12243117
    Abstract: The present disclosure relates to verifying an initial object estimation of an object. A two-dimensional (2D) image representative of an environment including one or more objects may be obtained. The 2D image may be inspected to detect edges of an object. The edges may be processed to verify or update an initial object estimation to increase the accuracy of an object detection result.
    Type: Grant
    Filed: November 15, 2023
    Date of Patent: March 4, 2025
    Assignee: Mujin, Inc.
    Inventors: Jinze Yu, Jose Jeronimo Moreira Rodrigues
  • Patent number: 12233548
    Abstract: A method for operating a robotic system including determining an initial pose of a target object based on imaging data; calculating a confidence measure associated with an accuracy of the initial pose; and determining that the confidence measure fails to satisfy a sufficiency condition; and deriving a motion plan accordingly for scanning an object identifier while transferring the target object from a start location to a task location.
    Type: Grant
    Filed: April 25, 2023
    Date of Patent: February 25, 2025
    Assignee: Mujin, Inc.
    Inventor: Rosen Diankov
  • Patent number: 12236609
    Abstract: A system and method for processing candidate edges are presented. The method may be performed when spatial structure information and 2D image information are stored. The method may include identifying, based on the spatial structure information, a 3D location that represents a corner of an object structure; identifying, based on the spatial structure information, a 3D vector that extends from the 3D location and is parallel with an edge; determining a 2D location within the 2D image information that corresponds to the 3D location; determining a 2D vector within the 2D image information that corresponds to the 3D vector; determining an edge detection region; identifying a cluster of candidate edges within the edge detection region that does not represent any object edge; identifying, as a 2D object edge, a candidate edge that is not part of the identified cluster; and performing segmentation based on the 2D object edge.
    Type: Grant
    Filed: July 12, 2022
    Date of Patent: February 25, 2025
    Assignee: MUJIN, INC.
    Inventors: Jinze Yu, Jose Jeronimo Moreira Rodrigues
  • Patent number: 12227374
    Abstract: A method for operating a robotic system includes determining a discretized object model representative of a target object; determining a discretized platform model representative of a task location; determining height measures based on real-time sensor data representative of the task location; and dynamically deriving a placement location based on (1) overlapping the discretized object model and the discretized platform model for stacking objects at the task location and (2) calculating a placement score associated with the overlapping based on the height measures.
    Type: Grant
    Filed: September 26, 2022
    Date of Patent: February 18, 2025
    Assignee: Mujin, Inc.
    Inventors: Rosen Nikolaev Diankov, Denys Kanunikov
  • Patent number: 12226907
    Abstract: End effectors for use with a robotic object-gripping system, and related systems and methods, are disclosed herein. In some embodiments, the end effector includes a first mounting structure, a force sensor coupled to the first mounting structure, a second mounting structure coupled to the force sensor, and a gripper assembly coupled to the second mounting structure. The force sensor is beneath the longitudinal plane and is configured to measure forces along a vertical axis. The end effector also includes a first bracket coupled to the first mounting structure and a second bracket coupled to the second mounting structure. The first and second brackets are configured to connect to the connection tubes to isolate the connection tubes, and any forces therein, to a longitudinal direction between the first bracket and the second bracket, thereby reducing the noise on the force sensor from the connection tubes during operation.
    Type: Grant
    Filed: January 11, 2022
    Date of Patent: February 18, 2025
    Assignee: Mujin, Inc.
    Inventors: Hironori Mizoguchi, Yoshiki Kanemoto