Patents by Inventor Nareshkumar Rajkumar

Nareshkumar Rajkumar has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 12265910
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for sharing learned information among robots. In some implementations, a robot obtains sensor data indicating characteristics of an object. The robot determines a classification for the object and generates an embedding for the object using a machine learning model stored by the robot. The robot stores the generated embedding and data indicating the classification for the object. The robot sends the generated embedding and the data indicating the classification to a server system. The robot receives, from the server system, an embedding generated by a second robot and a corresponding classification. The robot stores the received embedding and the corresponding classification in the local cache of the robot. The robot may then use the information in the cache to identify objects.
    Type: Grant
    Filed: September 9, 2022
    Date of Patent: April 1, 2025
    Assignee: Google LLC
    Inventors: Nareshkumar Rajkumar, Patrick Leger, Nicolas Hudson, Krishna Shankar, Rainer Hessmer
  • Publication number: 20250018561
    Abstract: Methods and apparatus related to generating a model for an object encountered by a robot in its environment, where the object is one that the robot is unable to recognize utilizing existing models associated with the robot. The model is generated based on vision sensor data that captures the object from multiple vantages and that is captured by a vision sensor associated with the robot, such as a vision sensor coupled to the robot. The model may be provided for use by the robot in detecting the object and/or for use in estimating the pose of the object.
    Type: Application
    Filed: September 27, 2024
    Publication date: January 16, 2025
    Inventors: Kurt Konolige, Nareshkumar Rajkumar, Stefan Hinterstoisser
  • Patent number: 12165021
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media for evaluating robot learning. In some implementations, a system receives classification examples from a plurality of remote devices over a communication network. The classification examples can include (i) a data representation generated by a remote device based on sensor data captured by the remote device and (ii) a classification corresponding to the data representation. The system assigns quality scores to the classification examples based on a level of similarity of the data representations with other data representations. The system selects a subset of the classification examples based on the quality scores assigned to the classification examples. The system trains a machine learning model using the selected subset of the classification examples.
    Type: Grant
    Filed: May 4, 2021
    Date of Patent: December 10, 2024
    Assignee: Google LLC
    Inventors: Nareshkumar Rajkumar, Patrick Leger
  • Patent number: 12103178
    Abstract: Methods and apparatus related to generating a model for an object encountered by a robot in its environment, where the object is one that the robot is unable to recognize utilizing existing models associated with the robot. The model is generated based on vision sensor data that captures the object from multiple vantages and that is captured by a vision sensor associated with the robot, such as a vision sensor coupled to the robot. The model may be provided for use by the robot in detecting the object and/or for use in estimating the pose of the object.
    Type: Grant
    Filed: June 22, 2023
    Date of Patent: October 1, 2024
    Assignee: GOOGLE LLC
    Inventors: Kurt Konolige, Nareshkumar Rajkumar, Stefan Hinterstoisser
  • Publication number: 20230398683
    Abstract: Methods and apparatus related to generating a model for an object encountered by a robot in its environment, where the object is one that the robot is unable to recognize utilizing existing models associated with the robot. The model is generated based on vision sensor data that captures the object from multiple vantages and that is captured by a vision sensor associated with the robot, such as a vision sensor coupled to the robot. The model may be provided for use by the robot in detecting the object and/or for use in estimating the pose of the object.
    Type: Application
    Filed: June 22, 2023
    Publication date: December 14, 2023
    Inventors: Kurt Konolige, Nareshkumar Rajkumar, Stefan Hinterstoisser
  • Patent number: 11727593
    Abstract: Methods for annotating objects within image frames are disclosed. Information is obtained that represents a camera pose relative to a scene. The camera pose includes a position and a location of the camera relative to the scene. Data is obtained that represents multiple images, including a first image and a plurality of other images, being captured from different angles by the camera relative to the scene. A 3D pose of the object of interest is identified with respect to the camera pose in at least the first image. A 3D bounding region for the object of interest in the first image is defined, which indicates a volume that includes the object of interest. A location and orientation of the object of interest is determined in the other images based on the defined 3D bounding region of the object of interest and the camera pose in the other images.
    Type: Grant
    Filed: August 9, 2021
    Date of Patent: August 15, 2023
    Assignee: Google LLC
    Inventors: Kurt Konolige, Nareshkumar Rajkumar, Stefan Hinterstoisser, Paul Wohlhart
  • Patent number: 11691273
    Abstract: Methods and apparatus related to generating a model for an object encountered by a robot in its environment, where the object is one that the robot is unable to recognize utilizing existing models associated with the robot. The model is generated based on vision sensor data that captures the object from multiple vantages and that is captured by a vision sensor associated with the robot, such as a vision sensor coupled to the robot. The model may be provided for use by the robot in detecting the object and/or for use in estimating the pose of the object.
    Type: Grant
    Filed: November 5, 2021
    Date of Patent: July 4, 2023
    Assignee: X DEVELOPMENT LLC
    Inventors: Kurt Konolige, Nareshkumar Rajkumar, Stefan Hinterstoisser
  • Patent number: 11685048
    Abstract: Methods, systems, and apparatus, including computer-readable media storing executable instructions, for enhancing robot learning. In some implementations, a robot stores first embeddings generated using a first machine learning model, and the first embeddings include one or more first private embeddings that are not shared with other robots. The robot receives a second machine learning model from a server system over a communication network. The robot generates a second private embedding for each of the one or more first private embeddings using the second machine learning model. The robot adds the second private embeddings to the cache of the robot and removes the one or more first private embeddings from the cache of the robot.
    Type: Grant
    Filed: April 5, 2021
    Date of Patent: June 27, 2023
    Assignee: X Development LLC
    Inventors: Nareshkumar Rajkumar, Patrick Leger, Abhinav Gupta
  • Patent number: 11577396
    Abstract: Methods, apparatus, systems, and computer-readable media are provided for visually annotating rendered multi-dimensional representations of robot environments. In various implementations, an entity may be identified that is present with a telepresence robot in an environment. A measure of potential interest of a user in the entity may be calculated based on a record of one or more interactions between the user and one or more computing devices. In some implementations, the one or more interactions may be for purposes other than directly operating the telepresence robot. In various implementations, a multi-dimensional representation of the environment may be rendered as part of a graphical user interface operable by the user to control the telepresence robot. In various implementations, a visual annotation may be selectively rendered within the multi-dimensional representation of the environment in association with the entity based on the measure of potential interest.
    Type: Grant
    Filed: July 15, 2020
    Date of Patent: February 14, 2023
    Assignee: X DEVELOPMENT LLC
    Inventors: Kyle Moore, Vincent Dureau, Nareshkumar Rajkumar
  • Publication number: 20230004802
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for sharing learned information among robots. In some implementations, a robot obtains sensor data indicating characteristics of an object. The robot determines a classification for the object and generates an embedding for the object using a machine learning model stored by the robot. The robot stores the generated embedding and data indicating the classification for the object. The robot sends the generated embedding and the data indicating the classification to a server system. The robot receives, from the server system, an embedding generated by a second robot and a corresponding classification. The robot stores the received embedding and the corresponding classification in the local cache of the robot. The robot may then use the information in the cache to identify objects.
    Type: Application
    Filed: September 9, 2022
    Publication date: January 5, 2023
    Inventors: Nareshkumar Rajkumar, Patrick Leger, Nicolas Hudson, Krishna Shankar, Rainer Hessmer
  • Patent number: 11475291
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for sharing learned information among robots. In some implementations, a robot obtains sensor data indicating characteristics of an object. The robot determines a classification for the object and generates an embedding for the object using a machine learning model stored by the robot. The robot stores the generated embedding and data indicating the classification for the object. The robot sends the generated embedding and the data indicating the classification to a server system. The robot receives, from the server system, an embedding generated by a second robot and a corresponding classification. The robot stores the received embedding and the corresponding classification in the local cache of the robot. The robot may then use the information in the cache to identify objects.
    Type: Grant
    Filed: December 27, 2017
    Date of Patent: October 18, 2022
    Assignee: X Development LLC
    Inventors: Nareshkumar Rajkumar, Patrick Leger, Nicolas Hudson, Krishna Shankar, Rainer Hessmer
  • Publication number: 20220058419
    Abstract: Methods and apparatus related to generating a model for an object encountered by a robot in its environment, where the object is one that the robot is unable to recognize utilizing existing models associated with the robot. The model is generated based on vision sensor data that captures the object from multiple vantages and that is captured by a vision sensor associated with the robot, such as a vision sensor coupled to the robot. The model may be provided for use by the robot in detecting the object and/or for use in estimating the pose of the object.
    Type: Application
    Filed: November 5, 2021
    Publication date: February 24, 2022
    Inventors: Kurt Konolige, Nareshkumar Rajkumar, Stefan Hinterstoisser
  • Patent number: 11195041
    Abstract: Methods and apparatus related to generating a model for an object encountered by a robot in its environment, where the object is one that the robot is unable to recognize utilizing existing models associated with the robot. The model is generated based on vision sensor data that captures the object from multiple vantages and that is captured by a vision sensor associated with the robot, such as a vision sensor coupled to the robot. The model may be provided for use by the robot in detecting the object and/or for use in estimating the pose of the object.
    Type: Grant
    Filed: May 1, 2020
    Date of Patent: December 7, 2021
    Assignee: X DEVELOPMENT LLC
    Inventors: Kurt Konolige, Nareshkumar Rajkumar, Stefan Hinterstoisser
  • Patent number: 11170220
    Abstract: Methods, apparatus, systems, and computer-readable media are provided for delegating object type and/or pose detection to a plurality of “targeted object recognition modules.” In some implementations, a method may be provided that includes: operating an object recognition client to facilitate object recognition for a robot; receiving, by the object recognition client, sensor data indicative of an observed object in an environment; providing, by the object recognition client, to each of a plurality of remotely-hosted targeted object recognition modules, data indicative of the observed object; receiving, by the object recognition client, from one or more of the plurality of targeted object recognition modules, one or more inferences about an object type or pose of the observed object; and determining, by the object recognition client, information about the observed object, such as its object type and/or pose, based on the one or more inferences.
    Type: Grant
    Filed: January 14, 2020
    Date of Patent: November 9, 2021
    Assignee: X DEVELOPMENT LLC
    Inventors: Nareshkumar Rajkumar, Stefan Hinterstoisser
  • Patent number: 11151744
    Abstract: Methods for annotating objects within image frames are disclosed. Information is obtained that represents a camera pose relative to a scene. The camera pose includes a position and a location of the camera relative to the scene. Data is obtained that represents multiple images, including a first image and a plurality of other images, being captured from different angles by the camera relative to the scene. A 3D pose of the object of interest is identified with respect to the camera pose in at least the first image. A 3D bounding region for the object of interest in the first image is defined, which indicates a volume that includes the object of interest. A location and orientation of the object of interest is determined in the other images based on the defined 3D bounding region of the object of interest and the camera pose in the other images.
    Type: Grant
    Filed: September 16, 2019
    Date of Patent: October 19, 2021
    Assignee: X Development LLC
    Inventors: Kurt Konolige, Nareshkumar Rajkumar, Stefan Hinterstoisser, Paul Wohlhart
  • Publication number: 20210256424
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media for evaluating robot learning. In some implementations, a system receives classification examples from a plurality of remote devices over a communication network. The classification examples can include (i) a data representation generated by a remote device based on sensor data captured by the remote device and (ii) a classification corresponding to the data representation. The system assigns quality scores to the classification examples based on a level of similarity of the data representations with other data representations. The system selects a subset of the classification examples based on the quality scores assigned to the classification examples. The system trains a machine learning model using the selected subset of the classification examples.
    Type: Application
    Filed: May 4, 2021
    Publication date: August 19, 2021
    Inventors: Nareshkumar Rajkumar, Patrick Leger
  • Publication number: 20210220991
    Abstract: Methods, systems, and apparatus, including computer-readable media storing executable instructions, for enhancing robot learning. In some implementations, a robot stores first embeddings generated using a first machine learning model, and the first embeddings include one or more first private embeddings that are not shared with other robots. The robot receives a second machine learning model from a server system over a communication network. The robot generates a second private embedding for each of the one or more first private embeddings using the second machine learning model. The robot adds the second private embeddings to the cache of the robot and removes the one or more first private embeddings from the cache of the robot.
    Type: Application
    Filed: April 5, 2021
    Publication date: July 22, 2021
    Inventors: Nareshkumar Rajkumar, Patrick Leger, Abhinav Gupta
  • Patent number: 11017317
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media for evaluating robot learning. In some implementations, one or more computers receive object classification examples from a plurality of robots. Each object classification example includes (i) an embedding that a robot generated using a machine learning model, and (ii) an object classification corresponding to the embedding. The object classification examples are evaluated based on a similarity of the received embeddings with respect to other embeddings. A subset of the object classification examples is selected based on the evaluation of the quality of the embeddings. The subset of the object classification examples is distributed to the robots in the plurality of robots.
    Type: Grant
    Filed: December 27, 2017
    Date of Patent: May 25, 2021
    Assignee: X Development LLC
    Inventors: Nareshkumar Rajkumar, Patrick Leger
  • Patent number: 10967509
    Abstract: Methods, systems, and apparatus, including computer-readable media storing executable instructions, for enhancing robot learning. In some implementations, a robot stores first embeddings generated using a first machine learning model, and the first embeddings include one or more first private embeddings that are not shared with other robots. The robot receives a second machine learning model from a server system over a communication network. The robot generates a second private embedding for each of the one or more first private embeddings using the second machine learning model. The robot adds the second private embeddings to the cache of the robot and removes the one or more first private embeddings from the cache of the robot.
    Type: Grant
    Filed: June 24, 2020
    Date of Patent: April 6, 2021
    Assignee: X Development LLC
    Inventors: Nareshkumar Rajkumar, Patrick Leger, Abhinav Gupta
  • Patent number: RE49262
    Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer-readable storage medium, and including a method for providing content. The method comprises receiving a first login request from a first device used by a user, the request being associated with a first anonymous identifier associated with the first device, and determining a user tag for the user, that does not include any personally identifiable information associated with the user. The method further comprises receiving a second login request from a second different device used by the user, the request being associated with a second different anonymous identifier associated with the second different device, and storing an association between the user tag, the first anonymous identifier and the second different anonymous identifier. The method further comprises receiving a request for content from either the first or second different device and providing content in response to the request using the association.
    Type: Grant
    Filed: April 13, 2020
    Date of Patent: October 25, 2022
    Assignee: GOOGLE LLC
    Inventors: Vinod Kumar Ramachandran, Ping Wu, Nareshkumar Rajkumar