Patents by Inventor Nicolas Hudson
Nicolas Hudson has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20240078683Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium for predicting object pose. In one aspect, a method includes receiving an image of an object having one or more feature points; providing the image as an input to a neural network subsystem trained to receive images of objects and to generate an output including a heat map for each feature point; applying a differentiable transformation on each heat map to generate respective one or more feature coordinates for each feature point; providing the feature coordinates for each feature point as input to an object pose solver configured to compute a predicted object pose for the object, wherein the predicted object pose for the object specifies a position and an orientation of an object; and receiving, at the output of the object pose solver, a predicted object pose for the object in the image.Type: ApplicationFiled: April 5, 2023Publication date: March 7, 2024Inventors: Mrinal Kalakrishnan, Adrian Ling Hin Li, Nicolas Hudson
-
Publication number: 20230405812Abstract: Methods, apparatus, and computer-readable media for determining and utilizing human corrections to robot actions. In some implementations, in response to determining a human correction of a robot action, a correction instance is generated that includes sensor data, captured by one or more sensors of the robot, that is relevant to the corrected action. The correction instance can further include determined incorrect parameter(s) utilized in performing the robot action and/or correction information that is based on the human correction. The correction instance can be utilized to generate training example(s) for training one or model(s), such as neural network model(s), corresponding to those used in determining the incorrect parameter(s).Type: ApplicationFiled: August 29, 2023Publication date: December 21, 2023Inventors: Nicolas Hudson, Devesh Yamparala
-
Patent number: 11780083Abstract: Methods, apparatus, and computer-readable media for determining and utilizing human corrections to robot actions. In some implementations, in response to determining a human correction of a robot action, a correction instance is generated that includes sensor data, captured by one or more sensors of the robot, that is relevant to the corrected action. The correction instance can further include determined incorrect parameter(s) utilized in performing the robot action and/or correction information that is based on the human correction. The correction instance can be utilized to generate training example(s) for training one or model(s), such as neural network model(s), corresponding to those used in determining the incorrect parameter(s). In various implementations, the training is based on correction instances from multiple robots. After a revised version of a model is generated, the revised version can thereafter be utilized by one or more of the multiple robots.Type: GrantFiled: November 5, 2021Date of Patent: October 10, 2023Assignee: GOOGLE LLCInventors: Nicolas Hudson, Devesh Yamparala
-
Publication number: 20230303193Abstract: A robot system includes: an upper body section including one or more end-effectors; a lower body section including one or more legs; and an intermediate body section coupling the upper and lower body sections. An upper body control system operates at least one of the end-effectors. The intermediate body section experiences a first intermediate body linear force and/or moment based on an end-effector force acting on the at least one end-effector. A lower body control system operates the one or more legs. The one or more legs experience respective surface reaction forces. The intermediate body section experiences a second intermediate body linear force and/or moment based on the surface reaction forces. The lower body control system operates the one or more legs so that the second intermediate body linear force balances the first intermediate linear force and the second intermediate body moment balances the first intermediate body moment.Type: ApplicationFiled: April 26, 2023Publication date: September 28, 2023Inventors: Kevin Blankespoor, Benjamin Stephens, Nicolas Hudson, Yeuhi Abe, Jennifer Barry
-
Publication number: 20230281422Abstract: Methods, apparatus, and computer-readable media for determining and utilizing corrections to robot actions. Some implementations are directed to updating a local features model of a robot in response to determining a human correction of an action performed by the robot. The local features model is used to determine, based on an embedding generated over a corresponding neural network model, one or more features that are most similar to the generated embedding. Updating the local features model in response to a human correction can include updating a feature embedding, of the local features model, that corresponds to the human correction. Adjustment(s) to the features model can immediately improve robot performance without necessitating retraining of the corresponding neural network model.Type: ApplicationFiled: April 27, 2023Publication date: September 7, 2023Inventors: Krishna Shankar, Nicolas Hudson, Alexander Toshev
-
Patent number: 11667343Abstract: A robot system includes: an upper body section including one or more end-effectors; a lower body section including one or more legs; and an intermediate body section coupling the upper and lower body sections. An upper body control system operates at least one of the end-effectors. The intermediate body section experiences a first intermediate body linear force and/or moment based on an end-effector force acting on the at least one end-effector. A lower body control system operates the one or more legs. The one or more legs experience respective surface reaction forces. The intermediate body section experiences a second intermediate body linear force and/or moment based on the surface reaction forces. The lower body control system operates the one or more legs so that the second intermediate body linear force balances the first intermediate linear force and the second intermediate body moment balances the first intermediate body moment.Type: GrantFiled: June 6, 2019Date of Patent: June 6, 2023Assignee: Boston Dynamics, Inc.Inventors: Kevin Blankespoor, Benjamin Stephens, Nicolas Hudson, Yeuhi Abe, Jennifer Barry
-
Patent number: 11640517Abstract: Methods, apparatus, and computer-readable media for determining and utilizing corrections to robot actions. Some implementations are directed to updating a local features model of a robot in response to determining a human correction of an action performed by the robot. The local features model is used to determine, based on an embedding generated over a corresponding neural network model, one or more features that are most similar to the generated embedding. Updating the local features model in response to a human correction can include updating a feature embedding, of the local features model, that corresponds to the human correction. Adjustment(s) to the features model can immediately improve robot performance without necessitating retraining of the corresponding neural network model.Type: GrantFiled: August 30, 2021Date of Patent: May 2, 2023Assignee: X DEVELOPMENT LLCInventors: Krishna Shankar, Nicolas Hudson, Alexander Toshev
-
Patent number: 11625852Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium for predicting object pose. In one aspect, a method includes receiving an image of an object having one or more feature points; providing the image as an input to a neural network subsystem trained to receive images of objects and to generate an output including a heat map for each feature point; applying a differentiable transformation on each heat map to generate respective one or more feature coordinates for each feature point; providing the feature coordinates for each feature point as input to an object pose solver configured to compute a predicted object pose for the object, wherein the predicted object pose for the object specifies a position and an orientation of an object; and receiving, at the output of the object pose solver, a predicted object pose for the object in the image.Type: GrantFiled: December 7, 2020Date of Patent: April 11, 2023Assignee: X Development LLCInventors: Mrinal Kalakrishnan, Adrian Ling Hin Li, Nicolas Hudson
-
Publication number: 20230004802Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for sharing learned information among robots. In some implementations, a robot obtains sensor data indicating characteristics of an object. The robot determines a classification for the object and generates an embedding for the object using a machine learning model stored by the robot. The robot stores the generated embedding and data indicating the classification for the object. The robot sends the generated embedding and the data indicating the classification to a server system. The robot receives, from the server system, an embedding generated by a second robot and a corresponding classification. The robot stores the received embedding and the corresponding classification in the local cache of the robot. The robot may then use the information in the cache to identify objects.Type: ApplicationFiled: September 9, 2022Publication date: January 5, 2023Inventors: Nareshkumar Rajkumar, Patrick Leger, Nicolas Hudson, Krishna Shankar, Rainer Hessmer
-
Patent number: 11475291Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for sharing learned information among robots. In some implementations, a robot obtains sensor data indicating characteristics of an object. The robot determines a classification for the object and generates an embedding for the object using a machine learning model stored by the robot. The robot stores the generated embedding and data indicating the classification for the object. The robot sends the generated embedding and the data indicating the classification to a server system. The robot receives, from the server system, an embedding generated by a second robot and a corresponding classification. The robot stores the received embedding and the corresponding classification in the local cache of the robot. The robot may then use the information in the cache to identify objects.Type: GrantFiled: December 27, 2017Date of Patent: October 18, 2022Assignee: X Development LLCInventors: Nareshkumar Rajkumar, Patrick Leger, Nicolas Hudson, Krishna Shankar, Rainer Hessmer
-
Publication number: 20220055209Abstract: Methods, apparatus, and computer-readable media for determining and utilizing human corrections to robot actions. In some implementations, in response to determining a human correction of a robot action, a correction instance is generated that includes sensor data, captured by one or more sensors of the robot, that is relevant to the corrected action. The correction instance can further include determined incorrect parameter(s) utilized in performing the robot action and/or correction information that is based on the human correction. The correction instance can be utilized to generate training example(s) for training one or model(s), such as neural network model(s), corresponding to those used in determining the incorrect parameter(s). In various implementations, the training is based on correction instances from multiple robots. After a revised version of a model is generated, the revised version can thereafter be utilized by one or more of the multiple robots.Type: ApplicationFiled: November 5, 2021Publication date: February 24, 2022Inventors: Nicolas Hudson, Devesh Yamparala
-
Publication number: 20210390371Abstract: Methods, apparatus, and computer-readable media for determining and utilizing corrections to robot actions. Some implementations are directed to updating a local features model of a robot in response to determining a human correction of an action performed by the robot. The local features model is used to determine, based on an embedding generated over a corresponding neural network model, one or more features that are most similar to the generated embedding. Updating the local features model in response to a human correction can include updating a feature embedding, of the local features model, that corresponds to the human correction. Adjustment(s) to the features model can immediately improve robot performance without necessitating retraining of the corresponding neural network model.Type: ApplicationFiled: August 30, 2021Publication date: December 16, 2021Inventors: Krishna Shankar, Nicolas Hudson, Alexander Toshev
-
Patent number: 11198217Abstract: Methods, apparatus, and computer-readable media for determining and utilizing human corrections to robot actions. In some implementations, in response to determining a human correction of a robot action, a correction instance is generated that includes sensor data, captured by one or more sensors of the robot, that is relevant to the corrected action. The correction instance can further include determined incorrect parameter(s) utilized in performing the robot action and/or correction information that is based on the human correction. The correction instance can be utilized to generate training example(s) for training one or model(s), such as neural network model(s), corresponding to those used in determining the incorrect parameter(s). In various implementations, the training is based on correction instances from multiple robots. After a revised version of a model is generated, the revised version can thereafter be utilized by one or more of the multiple robots.Type: GrantFiled: December 27, 2019Date of Patent: December 14, 2021Assignee: X DEVELOPMENT LLCInventors: Nicolas Hudson, Devesh Yamparala
-
Patent number: 11106967Abstract: Methods, apparatus, and computer-readable media for determining and utilizing corrections to robot actions. Some implementations are directed to updating a local features model of a robot in response to determining a human correction of an action performed by the robot. The local features model is used to determine, based on an embedding generated over a corresponding neural network model, one or more features that are most similar to the generated embedding. Updating the local features model in response to a human correction can include updating a feature embedding, of the local features model, that corresponds to the human correction. Adjustment(s) to the features model can immediately improve robot performance without necessitating retraining of the corresponding neural network model.Type: GrantFiled: July 3, 2017Date of Patent: August 31, 2021Assignee: X DEVELOPMENT LLCInventors: Krishna Shankar, Nicolas Hudson, Alexander Toshev
-
Patent number: 10861184Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium for predicting object pose. In one aspect, a method includes receiving an image of an object having one or more feature points; providing the image as an input to a neural network subsystem trained to receive images of objects and to generate an output including a heat map for each feature point; applying a differentiable transformation on each heat map to generate respective one or more feature coordinates for each feature point; providing the feature coordinates for each feature point as input to an object pose solver configured to compute a predicted object pose for the object, wherein the predicted object pose for the object specifies a position and an orientation of an object; and receiving, at the output of the object pose solver, a predicted object pose for the object in the image.Type: GrantFiled: January 19, 2017Date of Patent: December 8, 2020Assignee: X Development LLCInventors: Mrinal Kalakrishnan, Adrian Ling Hin Li, Nicolas Hudson
-
Patent number: 10853646Abstract: Methods, apparatus, systems, and computer-readable media are provided for generating spatial affordances for an object, in an environment of a robot, and utilizing the generated spatial affordances in one or more robotics applications directed to the object. Various implementations relate to applying vision data as input to a trained machine learning model, processing the vision data using the trained machine learning model to generate output defining one or more spatial affordances for an object captured by the vision data, and controlling one or more actuators of a robot based on the generated output. Various implementations additionally or alternatively relate to training such a machine learning model.Type: GrantFiled: June 26, 2019Date of Patent: December 1, 2020Assignee: X DEVELOPMENT LLCInventors: Adrian Li, Nicolas Hudson, Aaron Edsinger
-
Publication number: 20200130176Abstract: Methods, apparatus, and computer-readable media for determining and utilizing human corrections to robot actions. In some implementations, in response to determining a human correction of a robot action, a correction instance is generated that includes sensor data, captured by one or more sensors of the robot, that is relevant to the corrected action. The correction instance can further include determined incorrect parameter(s) utilized in performing the robot action and/or correction information that is based on the human correction. The correction instance can be utilized to generate training example(s) for training one or model(s), such as neural network model(s), corresponding to those used in determining the incorrect parameter(s). In various implementations, the training is based on correction instances from multiple robots. After a revised version of a model is generated, the revised version can thereafter be utilized by one or more of the multiple robots.Type: ApplicationFiled: December 27, 2019Publication date: April 30, 2020Inventors: Nicolas Hudson, Devesh Yamparala
-
Patent number: 10562181Abstract: Methods, apparatus, and computer-readable media for determining and utilizing human corrections to robot actions. In some implementations, in response to determining a human correction of a robot action, a correction instance is generated that includes sensor data, captured by one or more sensors of the robot, that is relevant to the corrected action. The correction instance can further include determined incorrect parameter(s) utilized in performing the robot action and/or correction information that is based on the human correction. The correction instance can be utilized to generate training example(s) for training one or model(s), such as neural network model(s), corresponding to those used in determining the incorrect parameter(s). In various implementations, the training is based on correction instances from multiple robots. After a revised version of a model is generated, the revised version can thereafter be utilized by one or more of the multiple robots.Type: GrantFiled: July 3, 2017Date of Patent: February 18, 2020Assignee: X Development LLCInventors: Nicolas Hudson, Devesh Yamparala
-
Publication number: 20190283822Abstract: A robot system includes: an upper body section including one or more end-effectors; a lower body section including one or more legs; and an intermediate body section coupling the upper and lower body sections. An upper body control system operates at least one of the end-effectors. The intermediate body section experiences a first intermediate body linear force and/or moment based on an end-effector force acting on the at least one end-effector. A lower body control system operates the one or more legs. The one or more legs experience respective surface reaction forces. The intermediate body section experiences a second intermediate body linear force and/or moment based on the surface reaction forces. The lower body control system operates the one or more legs so that the second intermediate body linear force balances the first intermediate linear force and the second intermediate body moment balances the first intermediate body moment.Type: ApplicationFiled: June 6, 2019Publication date: September 19, 2019Applicant: Boston Dynamics, Inc.Inventors: Kevin Blankespoor, Benjamin Stephens, Nicolas Hudson, Yeuhi Abe, Jennifer Barry
-
Patent number: 10351189Abstract: A robot system includes: an upper body section including one or more end-effectors; a lower body section including one or more legs; and an intermediate body section coupling the upper and lower body sections. An upper body control system operates at least one of the end-effectors. The intermediate body section experiences a first intermediate body linear force and/or moment based on an end-effector force acting on the at least one end-effector. A lower body control system operates the one or more legs. The one or more legs experience respective surface reaction forces. The intermediate body section experiences a second intermediate body linear force and/or moment based on the surface reaction forces. The lower body control system operates the one or more legs so that the second intermediate body linear force balances the first intermediate linear force and the second intermediate body moment balances the first intermediate body moment.Type: GrantFiled: December 13, 2016Date of Patent: July 16, 2019Assignee: Boston Dynamics, Inc.Inventors: Kevin Blankespoor, Benjamin Stephens, Nicolas Hudson, Yeuhi Abe, Jennifer Barry