Patents by Inventor Mrinal Kalakrishnan
Mrinal Kalakrishnan has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 11951622Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a generator neural network to adapt input images.Type: GrantFiled: March 23, 2022Date of Patent: April 9, 2024Assignee: Google LLCInventors: Paul Wohlhart, Stephen James, Mrinal Kalakrishnan, Konstantinos Bousmalis
-
Publication number: 20240078683Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium for predicting object pose. In one aspect, a method includes receiving an image of an object having one or more feature points; providing the image as an input to a neural network subsystem trained to receive images of objects and to generate an output including a heat map for each feature point; applying a differentiable transformation on each heat map to generate respective one or more feature coordinates for each feature point; providing the feature coordinates for each feature point as input to an object pose solver configured to compute a predicted object pose for the object, wherein the predicted object pose for the object specifies a position and an orientation of an object; and receiving, at the output of the object pose solver, a predicted object pose for the object in the image.Type: ApplicationFiled: April 5, 2023Publication date: March 7, 2024Inventors: Mrinal Kalakrishnan, Adrian Ling Hin Li, Nicolas Hudson
-
Patent number: 11625852Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium for predicting object pose. In one aspect, a method includes receiving an image of an object having one or more feature points; providing the image as an input to a neural network subsystem trained to receive images of objects and to generate an output including a heat map for each feature point; applying a differentiable transformation on each heat map to generate respective one or more feature coordinates for each feature point; providing the feature coordinates for each feature point as input to an object pose solver configured to compute a predicted object pose for the object, wherein the predicted object pose for the object specifies a position and an orientation of an object; and receiving, at the output of the object pose solver, a predicted object pose for the object in the image.Type: GrantFiled: December 7, 2020Date of Patent: April 11, 2023Assignee: X Development LLCInventors: Mrinal Kalakrishnan, Adrian Ling Hin Li, Nicolas Hudson
-
Patent number: 11615291Abstract: Methods, apparatus, and computer readable media related to combining and/or training one or more neural network modules based on version identifier(s) assigned to the neural network module(s). Some implementations are directed to using version identifiers of neural network modules in determining whether and/or how to combine multiple neural network modules to generate a combined neural network model for use by a robot and/or other apparatus. Some implementations are additionally or alternatively directed to assigning a version identifier to an endpoint of a neural network module based on one or more other neural network modules to which the neural network module is joined during training of the neural network module.Type: GrantFiled: July 1, 2020Date of Patent: March 28, 2023Assignee: X DEVELOPMENT LLCInventors: Adrian Li, Mrinal Kalakrishnan
-
Patent number: 11610153Abstract: Utilizing at least one existing policy (e.g. a manually engineered policy) for a robotic task, in generating reinforcement learning (RL) data that can be used in training an RL policy for an instance of RL of the robotic task. The existing policy can be one that, standing alone, will not generate data that is compatible with the instance of RL for the robotic task. In contrast, the generated RL data is compatible with RL for the robotic task at least by virtue of it including state data that is in a state space of the RL for the robotic task, and including actions that are in the action space of the RL for the robotic task. The generated RL data can be used in at least some of the initial training for the RL policy using reinforcement learning.Type: GrantFiled: December 30, 2019Date of Patent: March 21, 2023Assignee: X DEVELOPMENT LLCInventors: Alexander Herzog, Adrian Li, Mrinal Kalakrishnan, Benjamin Holson
-
Patent number: 11607807Abstract: Training and/or use of a machine learning model for placement of an object secured by an end effector of a robot. A trained machine learning model can be used to process: (1) a current image, captured by a vision component of a robot, that captures an end effector securing an object; (2) a candidate end effector action that defines a candidate motion of the end effector; and (3) a target placement input that indicates a target placement location for the object. Based on the processing, a prediction can be generated that indicates likelihood of successful placement of the object in the target placement location with application of the motion defined by the candidate end effector action. At many iterations, the candidate end effector action with the highest probability is selected and control commands provided to cause the end effector to move in conformance with the corresponding end effector action.Type: GrantFiled: April 14, 2021Date of Patent: March 21, 2023Assignee: X DEVELOPMENT LLCInventors: Seyed Mohammad Khansari Zadeh, Mrinal Kalakrishnan, Paul Wohlhart
-
Patent number: 11607802Abstract: Generating and utilizing action image(s) that represent a candidate pose (e.g., a candidate end effector pose), in determining whether to utilize the candidate pose in performance of a robotic task. The action image(s) and corresponding current image(s) can be processed, using a trained critic network, to generate a value that indicates a probability of success of the robotic task if component(s) of the robot are traversed to the particular pose. When the value satisfies one or more conditions (e.g., satisfies a threshold), the robot can be controlled to cause the component(s) to traverse to the particular pose in performing the robotic task.Type: GrantFiled: May 28, 2020Date of Patent: March 21, 2023Assignee: X DEVELOPMENT LLCInventors: Seyed Mohammad Khansari Zadeh, Daniel Kappler, Jianlan Luo, Jeffrey Bingham, Mrinal Kalakrishnan
-
Patent number: 11571809Abstract: Techniques are described herein for robotic control using value distributions. In various implementations, as part of performing a robotic task, state data associated with the robot in an environment may be generated based at least in part on vision data captured by a vision component of the robot. A plurality of candidate actions may be sampled, e.g., from continuous action space. A trained critic neural network model that represents a learned value function may be used to process a plurality of state-action pairs to generate a corresponding plurality of value distributions. Each state-action pair may include the state data and one of the plurality of sampled candidate actions. The state-action pair corresponding to the value distribution that satisfies one or more criteria may be selected from the plurality of state-action pairs. The robot may then be controlled to implement the sampled candidate action of the selected state-action pair.Type: GrantFiled: September 11, 2020Date of Patent: February 7, 2023Assignee: X DEVELOPMENT LLCInventors: Cristian Bodnar, Adrian Li, Karol Hausman, Peter Pastor Sampedro, Mrinal Kalakrishnan
-
Patent number: 11565401Abstract: Methods and apparatus related to receiving a request that includes robot instructions and/or environmental parameters, operating each of a plurality of robots based on the robot instructions and/or in an environment configured based on the environmental parameters, and storing data generated by the robots during the operating. In some implementations, at least part of the stored data that is generated by the robots is provided in response to the request and/or additional data that is generated based on the stored data is provided in response to the request.Type: GrantFiled: March 22, 2021Date of Patent: January 31, 2023Assignee: X DEVELOPMENT LLCInventors: Peter Pastor Sampedro, Mrinal Kalakrishnan, Ali Yahya Valdovinos, Adrian Li, Kurt Konolige, Vincent Dureau
-
Publication number: 20220245503Abstract: Implementations disclosed herein relate to utilizing at least one existing manually engineered policy, for a robotic task, in training an RL policy model that can be used to at least selectively replace a portion of the engineered policy. The RL policy model can be trained for replacing a portion of a robotic task and can be trained based on data from episodes of attempting performance of the robotic task, including episodes in which the portion is performed based on the engineered policy and/or other portion(s) are performed based on the engineered policy. Once trained, the RL policy model can be used, at least selectively and in lieu of utilization of the engineered policy, to perform the portion of robotic task, while other portion(s) of the robotic task are performed utilizing the engineered policy and/or other similarly trained (but distinct) RL policy model(s).Type: ApplicationFiled: January 29, 2021Publication date: August 4, 2022Inventors: Adrian Li, Benjamin Holson, Alexander Herzog, Mrinal Kalakrishnan
-
Publication number: 20220215208Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a generator neural network to adapt input images.Type: ApplicationFiled: March 23, 2022Publication date: July 7, 2022Inventors: Paul Wohlhart, Stephen James, Mrinal Kalakrishnan, Konstantinos Bousmalis
-
Patent number: 11341364Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training an action selection neural network that is used to control a robotic agent interacting with a real-world environment.Type: GrantFiled: September 20, 2018Date of Patent: May 24, 2022Assignee: Google LLCInventors: Konstantinos Bousmalis, Alexander Irpan, Paul Wohlhart, Yunfei Bai, Mrinal Kalakrishnan, Julian Ibarz, Sergey Vladimir Levine, Kurt Konolige, Vincent O. Vanhoucke, Matthew Laurance Kelcey
-
Patent number: 11325252Abstract: Deep machine learning methods and apparatus related to the manipulation of an object by an end effector of a robot are described herein. Some implementations relate to training an action prediction network to predict a probability density which can include candidate actions of successful grasps by the end effector given an input image. Some implementations are directed to utilization of an action prediction network to visually servo a grasping end effector of a robot to achieve a successful grasp of an object by the grasping end effector.Type: GrantFiled: September 13, 2019Date of Patent: May 10, 2022Assignee: X DEVELOPMENT LLCInventors: Adrian Li, Peter Pastor Sampedro, Mengyuan Yan, Mrinal Kalakrishnan
-
Patent number: 11314987Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a generator neural network to adapt input images.Type: GrantFiled: November 22, 2019Date of Patent: April 26, 2022Assignee: X Development LLCInventors: Paul Wohlhart, Stephen James, Mrinal Kalakrishnan, Konstantinos Bousmalis
-
Publication number: 20220105624Abstract: Techniques are disclosed that enable training a meta-learning model, for use in causing a robot to perform a task, using imitation learning as well as reinforcement learning. Some implementations relate to training the meta-learning model using imitation learning based on one or more human guided demonstrations of the task. Additional or alternative implementations relate to training the meta-learning model using reinforcement learning based on trials of the robot attempting to perform the task. Further implementations relate to using the trained meta-learning model to few shot (or one shot) learn a new task based on a human guided demonstration of the new task.Type: ApplicationFiled: January 23, 2020Publication date: April 7, 2022Inventors: Mrinal Kalakrishnan, Yunfei Bai, Paul Wohlhart, Eric Jang, Chelsea Finn, Seyed Mohammad Khansari Zadeh, Sergey Levine, Allan Zhou, Alexander Herzog, Daniel Kappler
-
Patent number: 11188821Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, of training a global policy neural network. One of the methods includes initializing an instance of the robotic task for multiple local workers, generating a trajectory of state-action pairs by selecting actions to be performed by the robotic agent while performing the instance of the robotic task, optimizing a local policy controller on the trajectory, generating an optimized trajectory using the optimized local controller, and storing the optimized trajectory in a replay memory associated with the local worker. The method includes sampling, for multiple global workers, an optimized trajectory from one of one or more replay memories associated with the global worker, and training the replica of the global policy neural network maintained by the global worker on the sampled optimized trajectory to determine delta values for the parameters of the global policy neural network.Type: GrantFiled: September 15, 2017Date of Patent: November 30, 2021Assignee: X Development LLCInventors: Mrinal Kalakrishnan, Ali Hamid Yahya Valdovinos, Adrian Ling Hin Li, Yevgen Chebotar, Sergey Vladimir Levine
-
Patent number: 11179847Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for a system configured to plan actions to be performed by a robotic agent interacting with an environment to accomplish an objective by determining an optimized trajectory of state—action pairs for accomplishing the objective. The system maintains a current optimized trajectory and a current trust region radius, and optimizes a localized objective within the current trust region radius of the current optimized trajectory to determine a candidate updated optimized trajectory. The system determines whether the candidate updated optimized trajectory improves over the current optimized trajectory. In response to determining that the candidate updated optimized trajectory improves over the current optimized trajectory, the system updates the current optimized trajectory to the candidate updated optimized trajectory and updates the current trust region radius.Type: GrantFiled: October 12, 2017Date of Patent: November 23, 2021Assignee: Google LLCInventors: Mrinal Kalakrishnan, Vikas Sindhwani
-
Publication number: 20210229276Abstract: Training and/or use of a machine learning model for placement of an object secured by an end effector of a robot. A trained machine learning model can be used to process: (1) a current image, captured by a vision component of a robot, that captures an end effector securing an object; (2) a candidate end effector action that defines a candidate motion of the end effector; and (3) a target placement input that indicates a target placement location for the object. Based on the processing, a prediction can be generated that indicates likelihood of successful placement of the object in the target placement location with application of the motion defined by the candidate end effector action. At many iterations, the candidate end effector action with the highest probability is selected and control commands provided to cause the end effector to move in conformance with the corresponding end effector action.Type: ApplicationFiled: April 14, 2021Publication date: July 29, 2021Inventors: Seyed Mohammad Khansari Zadeh, Mrinal Kalakrishnan, Paul Wohlhart
-
Patent number: 11007642Abstract: Training and/or use of a machine learning model for placement of an object secured by an end effector of a robot. A trained machine learning model can be used to process: (1) a current image, captured by a vision component of a robot, that captures an end effector securing an object; (2) a candidate end effector action that defines a candidate motion of the end effector; and (3) a target placement input that indicates a target placement location for the object. Based on the processing, a prediction can be generated that indicates likelihood of successful placement of the object in the target placement location with application of the motion defined by the candidate end effector action. At many iterations, the candidate end effector action with the highest probability is selected and control commands provided to cause the end effector to move in conformance with the corresponding end effector action.Type: GrantFiled: October 23, 2018Date of Patent: May 18, 2021Assignee: X DEVELOPMENT LLCInventors: Seyed Mohammad Khansari Zadeh, Mrinal Kalakrishnan, Paul Wohlhart
-
Patent number: 10981270Abstract: Methods and apparatus related to receiving a request that includes robot instructions and/or environmental parameters, operating each of a plurality of robots based on the robot instructions and/or in an environment configured based on the environmental parameters, and storing data generated by the robots during the operating. In some implementations, at least part of the stored data that is generated by the robots is provided in response to the request and/or additional data that is generated based on the stored data is provided in response to the request.Type: GrantFiled: August 2, 2019Date of Patent: April 20, 2021Assignee: X DEVELOPMENT LLCInventors: Peter Pastor Sampedro, Mrinal Kalakrishnan, Ali Yahya Valdovinos, Adrian Li, Kurt Konolige, Vincent Dureau