Patents by Inventor Jonathan Tremblay

Jonathan Tremblay has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 12194632
    Abstract: In at least one embodiment, under the control of a robotic control system, a gripper on a robot is positioned to grasp a 3-dimensional object. In at least one embodiment, the relative position of the object and the gripper is determined, at least in part, by using a camera mounted on the gripper.
    Type: Grant
    Filed: October 10, 2023
    Date of Patent: January 14, 2025
    Assignee: NVIDIA Corporation
    Inventors: Shariq Iqbal, Jonathan Tremblay, Thang Hong To, Jia Cheng, Erik Leitch, Duncan J. McKay, Stanley Thomas Birchfield
  • Patent number: 12175703
    Abstract: Apparatuses, systems, and techniques to determine a pose and relative dimensions of an object from an image. In at least one embodiment, a pose and relative dimensions of an object are determined from an image based at least in part on, for example, features of the image.
    Type: Grant
    Filed: September 9, 2021
    Date of Patent: December 24, 2024
    Assignee: NVIDIA Corporation
    Inventors: Stanley Thomas Birchfield, Jonathan Tremblay, Yunzhi Lin, Stephen Walter Tyree
  • Patent number: 12109701
    Abstract: A robot is controlled using a combination of model-based and model-free control methods. In some examples, the model-based method uses a physical model of the environment around the robot to guide the robot. The physical model is oriented using a perception system such as a camera. Characteristics of the perception system may be are used to determine an uncertainty for the model. Based at least in part on this uncertainty, the system transitions from the model-based method to a model-free method where, in some embodiments, information provided directly from the perception system is used to direct the robot without reliance on the physical model.
    Type: Grant
    Filed: February 3, 2020
    Date of Patent: October 8, 2024
    Assignee: NVIDIA Corporation
    Inventors: Jonathan Tremblay, Dieter Fox, Michelle Lee, Carlos Florensa, Nathan Donald Ratliff, Animesh Garg, Fabio Tozeto Ramos
  • Patent number: 12017352
    Abstract: Apparatuses, systems, and techniques to map coordinates in task space to a set of joint angles of an articulated robot. In at least one embodiment, a neural network is trained to map task-space coordinates to joint space coordinates of a robot by simulating a plurality of robots at various joint angles, and determining the position of their respective manipulators in task space.
    Type: Grant
    Filed: February 16, 2021
    Date of Patent: June 25, 2024
    Assignee: NVIDIA CORPORATION
    Inventors: Visak Chadalavada Vijay Kumar, David Hoeller, Balakumar Sundaralingam, Jonathan Tremblay, Stanley Thomas Birchfield
  • Publication number: 20240169563
    Abstract: Apparatuses, systems, and techniques for constructing a data structure to store a shape of an object based at least in part on a portion of multiple images, and obtaining poses of the object by tracking a pose of the object through the multiple images based at least in part on the data structure. Optionally, the poses may be used to generate a plan for a path of a device to travel, generate a rendering of at least a portion of a Mixed Reality (“MR”) display to be viewed by a user, and/or the like.
    Type: Application
    Filed: November 15, 2023
    Publication date: May 23, 2024
    Inventors: Bowen Wen, Jonathan Tremblay, Valts Blukis, Jan Kautz, Stanley Thomas Birchfield
  • Publication number: 20240153196
    Abstract: Apparatuses, systems, and techniques to generate an image of one or more objects. In at least one embodiment, an image of one or more objects is generated using a neural network based on, for example, a representation of a scene.
    Type: Application
    Filed: July 5, 2023
    Publication date: May 9, 2024
    Inventors: Valts Blukis, Taeyeop Lee, Jonathan Tremblay, Bowen Wen, Dieter Fox, Stanley Thomas Birchfield
  • Publication number: 20240123620
    Abstract: Apparatuses, systems, and techniques to generate and select grasp proposals. In at least one embodiment, grasp proposals are generated and selected using one or more neural networks, based on, for example, a latent code corresponding to an object.
    Type: Application
    Filed: July 6, 2023
    Publication date: April 18, 2024
    Inventors: Jonathan Tremblay, Stanley Thomas Birchfield, Valts Blukis, Bowen Wen, Dieter Fox, Taeyeop Lee
  • Patent number: 11941719
    Abstract: Various embodiments enable a robot, or other autonomous or semi-autonomous device or system, to receive data involving the performance of a task in the physical world. The data can be provided as input to a perception network to infer a set of percepts about the task, which can correspond to relationships between objects observed during the performance. The percepts can be provided as input to a plan generation network, which can infer a set of actions as part of a plan. Each action can correspond to one of the observed relationships. The plan can be reviewed and any corrections made, either manually or through another demonstration of the task. Once the plan is verified as correct, the plan (and any related data) can be provided as input to an execution network that can infer instructions to cause the robot, and/or another robot, to perform the task.
    Type: Grant
    Filed: January 23, 2019
    Date of Patent: March 26, 2024
    Assignee: NVIDIA Corporation
    Inventors: Jonathan Tremblay, Stan Birchfield, Stephen Tyree, Thang To, Jan Kautz, Artem Molchanov
  • Patent number: 11941899
    Abstract: Apparatuses, systems, and techniques generate poses of an object based on image data of the object obtained from a first viewpoint of the object and a second viewpoint of the object. The poses can be evaluated to determine a portion of the image data usable by an estimator to generate a pose of the object.
    Type: Grant
    Filed: May 26, 2021
    Date of Patent: March 26, 2024
    Assignee: NVIDIA Corporation
    Inventors: Jonathan Tremblay, Fabio Tozeto Ramos, Yuke Zhu, Anima Anandkumar, Guanya Shi
  • Publication number: 20240095077
    Abstract: Apparatuses, systems, and techniques to generate a prompt for one or more machine learning processes. In at least one embodiment, the machine learning process(es) generate(s) a plan to perform a task (identified in the prompt) that is to be performed by an agent (real world or virtual).
    Type: Application
    Filed: March 16, 2023
    Publication date: March 21, 2024
    Inventors: Ishika Singh, Arsalan Mousavian, Ankit Goyal, Danfei Xu, Jonathan Tremblay, Dieter Fox, Animesh Garg, Valts Blukis
  • Patent number: 11931909
    Abstract: Apparatuses, systems, and techniques generate poses of an object based on data of the object observed from a first viewpoint and a second viewpoint. The poses can be evaluated to determine a portion of the data usable by an estimator to generate a pose of the object.
    Type: Grant
    Filed: May 26, 2021
    Date of Patent: March 19, 2024
    Assignee: NVIDIA Corporation
    Inventors: Jonathan Tremblay, Fabio Tozeto Ramos, Yuke Zhu, Anima Anandkumar, Guanya Shi
  • Publication number: 20240042601
    Abstract: In at least one embodiment, under the control of a robotic control system, a gripper on a robot is positioned to grasp a 3-dimensional object. In at least one embodiment, the relative position of the object and the gripper is determined, at least in part, by using a camera mounted on the gripper.
    Type: Application
    Filed: October 10, 2023
    Publication date: February 8, 2024
    Inventors: Shariq Iqbal, Jonathan Tremblay, Thang Hong To, Jia Cheng, Erik Leitch, Duncan J. McKay, Stanley Thomas Birchfield
  • Publication number: 20240005547
    Abstract: Apparatuses, systems, and techniques to determined a pose of an object from a plurality of images. In at least one embodiment, the pose of an object is determined from at least two images of a video sequence using one or more neural networks, in which the neural network produces a distribution of pose information that is filtered to determine the current pose.
    Type: Application
    Filed: May 23, 2022
    Publication date: January 4, 2024
    Inventors: Yunzhi Lin, Jonathan Tremblay, Stephen Walter Tyree, Stanley Thomas Birchfield
  • Publication number: 20230415336
    Abstract: A robot device determines an error associated with equipment included in a data center environment. The robot device may compare the error to candidate errors for which the robot device is already trained to resolve. Based on a result of the comparison, the robot device may perform, in a control environment, candidate maintenance operations in association with resolving the error. The robot device may learn a set of actions associated with successfully resolving the error, based on performing the candidate maintenance operations. The robot device may perform maintenance operations associated with the error. Performing the maintenance operations may include applying the learned set of actions.
    Type: Application
    Filed: June 27, 2022
    Publication date: December 28, 2023
    Inventors: Siddha Ganju, Elad Mentovich, James Stephen Fields, JR., Ryan Kelsey Albright, Jonathan Tremblay, Stanley Thomas Birchfield
  • Patent number: 11833681
    Abstract: In at least one embodiment, under the control of a robotic control system, a gripper on a robot is positioned to grasp a 3-dimensional object. In at least one embodiment, the relative position of the object and the gripper is determined, at least in part, by using a camera mounted on the gripper.
    Type: Grant
    Filed: August 23, 2019
    Date of Patent: December 5, 2023
    Assignee: NVIDIA Corporation
    Inventors: Shariq Iqbal, Jonathan Tremblay, Thang Hong To, Jia Cheng, Erik Leitch, Duncan J. McKay, Stanley Thomas Birchfield
  • Patent number: 11830145
    Abstract: A manifold voxel mesh or surface mesh is manufacturable by carving a single block of material and a non-manifold mesh is not manufacturable. Conventional techniques for constructing or extracting a surface mesh from an input point cloud often produce a non-manifold voxel mesh. Similarly, extracting a surface mesh from a voxel mesh that includes non-manifold geometry produces a surface mesh that includes non-manifold geometry. To ensure that the surface mesh includes only manifold geometry, locations of the non-manifold geometry in the voxel mesh are detected and converted into manifold geometry. The result is a manifold voxel mesh from which a manifold surface mesh of the object may be extracted.
    Type: Grant
    Filed: September 20, 2021
    Date of Patent: November 28, 2023
    Assignee: NVIDIA Corporation
    Inventors: Kunal Gupta, Shalini De Mello, Charles Loop, Jonathan Tremblay, Stanley Thomas Birchfield
  • Patent number: 11715251
    Abstract: Training deep neural networks requires a large amount of labeled training data. Conventionally, labeled training data is generated by gathering real images that are manually labelled which is very time-consuming. Instead of manually labelling a training dataset, domain randomization technique is used generate training data that is automatically labeled. The generated training data may be used to train neural networks for object detection and segmentation (labelling) tasks. In an embodiment, the generated training data includes synthetic input images generated by rendering three-dimensional (3D) objects of interest in a 3D scene. In an embodiment, the generated training data includes synthetic input images generated by rendering 3D objects of interest on a 2D background image. The 3D objects of interest are objects that a neural network is trained to detect and/or label.
    Type: Grant
    Filed: October 21, 2021
    Date of Patent: August 1, 2023
    Assignee: NVIDIA Corporation
    Inventors: Jonathan Tremblay, Aayush Prakash, Mark A. Brophy, Varun Jampani, Cem Anil, Stanley Thomas Birchfield, Thang Hong To, David Jesus Acuna Marrero
  • Publication number: 20230104782
    Abstract: A manifold voxel mesh or surface mesh is manufacturable by carving a single block of material and a non-manifold mesh is not manufacturable. Conventional techniques for constructing or extracting a surface mesh from an input point cloud often produce a non-manifold voxel mesh. Similarly, extracting a surface mesh from a voxel mesh that includes non-manifold geometry produces a surface mesh that includes non-manifold geometry. To ensure that the surface mesh includes only manifold geometry, locations of the non-manifold geometry in the voxel mesh are detected and converted into manifold geometry. The result is a manifold voxel mesh from which a manifold surface mesh of the object may be extracted.
    Type: Application
    Filed: September 20, 2021
    Publication date: April 6, 2023
    Inventors: Kunal Gupta, Shalini De Mello, Charles Loop, Jonathan Tremblay, Stanley Thomas Birchfield
  • Publication number: 20230081641
    Abstract: A single two-dimensional (2D) image can be used as input to obtain a three-dimensional (3D) representation of the 2D image. This is done by extracting features from the 2D image by an encoder and determining a 3D representation of the 2D image utilizing a trained 2D convolutional neural network (CNN). Volumetric rendering is then run on the 3D representation to combine features within one or more viewing directions, and the combined features are provided as input to a multilayer perceptron (MLP) that predicts and outputs color (or multi-dimensional neural features) and density values for each point within the 3D representation. As a result, single-image inverse rendering may be performed using only a single 2D image as input to create a corresponding 3D representation of the scene in the single 2D image.
    Type: Application
    Filed: December 14, 2021
    Publication date: March 16, 2023
    Inventors: Koki Nagano, Eric Ryan Chan, Sameh Khamis, Shalini De Mello, Tero Tapani Karras, Orazio Gallo, Jonathan Tremblay
  • Publication number: 20220379484
    Abstract: Apparatuses, systems, and techniques generate poses of an object based on data of the object observed from a first viewpoint and a second viewpoint. The poses can be evaluated to determine a portion of the data usable by an estimator to generate a pose of the object.
    Type: Application
    Filed: May 26, 2021
    Publication date: December 1, 2022
    Inventors: Jonathan Tremblay, Fabio Tozeto Ramos, Yuke Zhu, Anima Anandkumar, Guanya Shi