Patents by Inventor David Jesus Acuna Marrero

David Jesus Acuna Marrero has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20250086922
    Abstract: Apparatuses, system, and techniques use one or more neural networks to generate a modified bounding box based, at least in part, on one or more second bounding boxes.
    Type: Application
    Filed: September 7, 2023
    Publication date: March 13, 2025
    Inventors: David Jesus Acuna Marrero, Rafid Mahmood, James Robert Lucas, Yuan-Hong Liao, Sanja Fidler
  • Publication number: 20250061153
    Abstract: A generative model can be used for generation of spatial layouts and graphs. Such a model can progressively grow these layouts and graphs based on local statistics, where nodes can represent spatial control points of the layout, and edges can represent segments or paths between nodes, such as may correspond to road segments. A generative model can utilize an encoder-decoder architecture where the encoder is a recurrent neural network (RNN) that encodes local incoming paths into a node and the decoder is another RNN that generates outgoing nodes and edges connecting an existing node to the newly generated nodes. Generation is done iteratively, and can finish once all nodes are visited or another end condition is satisfied. Such a model can generate layouts by additionally conditioning on a set of attributes, giving control to a user in generating the layout.
    Type: Application
    Filed: November 1, 2024
    Publication date: February 20, 2025
    Inventors: Hang Chu, Daiqing Li, David Jesus Acuna Marrero, Amlan Kar, Maria Shugrina, Ming-Yu Liu, Antonio Torralba Barriuso, Sanja Fidler
  • Publication number: 20250054288
    Abstract: Various examples relate to translating image labels from one domain (e.g., a synthetic domain) to another domain (e.g., a real-world domain) to improve model performance on real-world datasets and applications. Systems and methods are disclosed that provide an unsupervised label translator that may employ a generative adversarial network (GAN)-based approach. In contrast to conventional systems, the disclosed approach can employ a data-centric perspective that addresses systematic mismatches between datasets from different sources.
    Type: Application
    Filed: August 7, 2023
    Publication date: February 13, 2025
    Applicant: NVIDIA Corporation
    Inventors: Yuan-Hong LIAO, David Jesus ACUNA MARRERO, James LUCAS, Rafid MAHMOOD, Sanja FIDLER, Viraj Uday PRABHU
  • Patent number: 12141986
    Abstract: Various types of image analysis benefit from a multi-stream architecture that allows the analysis to consider shape data. A shape stream can process image data in parallel with a primary stream, where data from layers of a network in the primary stream is provided as input to a network of the shape stream. The shape data can be fused with the primary analysis data to produce more accurate output, such as to produce accurate boundary information when the shape data is used with semantic segmentation data produced by the primary stream. A gate structure can be used to connect the intermediate layers of the primary and shape streams, using higher level activations to gate lower level activations in the shape stream. Such a gate structure can help focus the shape stream on the relevant information and reduces any additional weight of the shape stream.
    Type: Grant
    Filed: June 12, 2023
    Date of Patent: November 12, 2024
    Assignee: Nvidia Corporation
    Inventors: David Jesus Acuna Marrero, Towaki Takikawa, Varun Jampani, Sanja Fidler
  • Publication number: 20230385687
    Abstract: Approaches for training data set size estimation for machine learning model systems and applications are described. Examples include a machine learning model training system that estimates target data requirements for training a machine learning model, given an approximate relationship between training data set size and model performance using one or more validation score estimation functions. To derive a validation score estimation function, a regression data set is generated from training data, and subsets of the regression data set are used to train the machine learning model. A validation score is computed for the subsets and used to compute regression function parameters to curve fit the selected regression function to the training data set. The validation score estimation function is then solved for and provides an output of an estimate of the number additional training samples needed for the validation score estimation function to meet or exceed a target validation score.
    Type: Application
    Filed: May 31, 2022
    Publication date: November 30, 2023
    Inventors: Rafid Reza Mahmood, James Robert Lucas, David Jesus Acuna Marrero, Daiqing Li, Jonah Philion, Jose Manuel Alvarez Lopez, Zhiding Yu, Sanja Fidler, Marc Law
  • Publication number: 20230342941
    Abstract: Various types of image analysis benefit from a multi-stream architecture that allows the analysis to consider shape data. A shape stream can process image data in parallel with a primary stream, where data from layers of a network in the primary stream is provided as input to a network of the shape stream. The shape data can be fused with the primary analysis data to produce more accurate output, such as to produce accurate boundary information when the shape data is used with semantic segmentation data produced by the primary stream. A gate structure can be used to connect the intermediate layers of the primary and shape streams, using higher level activations to gate lower level activations in the shape stream. Such a gate structure can help focus the shape stream on the relevant information and reduces any additional weight of the shape stream.
    Type: Application
    Filed: June 12, 2023
    Publication date: October 26, 2023
    Inventors: David Jesus Acuna Marrero, Towaki Takikawa, Varun Jampani, Sanja Fidler
  • Patent number: 11715251
    Abstract: Training deep neural networks requires a large amount of labeled training data. Conventionally, labeled training data is generated by gathering real images that are manually labelled which is very time-consuming. Instead of manually labelling a training dataset, domain randomization technique is used generate training data that is automatically labeled. The generated training data may be used to train neural networks for object detection and segmentation (labelling) tasks. In an embodiment, the generated training data includes synthetic input images generated by rendering three-dimensional (3D) objects of interest in a 3D scene. In an embodiment, the generated training data includes synthetic input images generated by rendering 3D objects of interest on a 2D background image. The 3D objects of interest are objects that a neural network is trained to detect and/or label.
    Type: Grant
    Filed: October 21, 2021
    Date of Patent: August 1, 2023
    Assignee: NVIDIA Corporation
    Inventors: Jonathan Tremblay, Aayush Prakash, Mark A. Brophy, Varun Jampani, Cem Anil, Stanley Thomas Birchfield, Thang Hong To, David Jesus Acuna Marrero
  • Publication number: 20230229919
    Abstract: In various examples, a generative model is used to synthesize datasets for use in training a downstream machine learning model to perform an associated task. The synthesized datasets may be generated by sampling a scene graph from a scene grammar—such as a probabilistic grammar— and applying the scene graph to the generative model to compute updated scene graphs more representative of object attribute distributions of real-world datasets. The downstream machine learning model may be validated against a real-world validation dataset, and the performance of the model on the real-world validation dataset may be used as an additional factor in further training or fine-tuning the generative model for generating the synthesized datasets specific to the task of the downstream machine learning model.
    Type: Application
    Filed: March 20, 2023
    Publication date: July 20, 2023
    Inventors: Amlan Kar, Aayush Prakash, Ming-Yu Liu, David Jesus Acuna Marrero, Antonio Torralba Barriuso, Sanja Fidler
  • Patent number: 11676284
    Abstract: Various types of image analysis benefit from a multi-stream architecture that allows the analysis to consider shape data. A shape stream can process image data in parallel with a primary stream, where data from layers of a network in the primary stream is provided as input to a network of the shape stream. The shape data can be fused with the primary analysis data to produce more accurate output, such as to produce accurate boundary information when the shape data is used with semantic segmentation data produced by the primary stream. A gate structure can be used to connect the intermediate layers of the primary and shape streams, using higher level activations to gate lower level activations in the shape stream. Such a gate structure can help focus the shape stream on the relevant information and reduces any additional weight of the shape stream.
    Type: Grant
    Filed: March 20, 2020
    Date of Patent: June 13, 2023
    Assignee: Nvidia Corporation
    Inventors: David Jesus Acuna Marrero, Towaki Takikawa, Varun Jampani, Sanja Fidler
  • Patent number: 11610115
    Abstract: In various examples, a generative model is used to synthesize datasets for use in training a downstream machine learning model to perform an associated task. The synthesized datasets may be generated by sampling a scene graph from a scene grammar—such as a probabilistic grammar—and applying the scene graph to the generative model to compute updated scene graphs more representative of object attribute distributions of real-world datasets. The downstream machine learning model may be validated against a real-world validation dataset, and the performance of the model on the real-world validation dataset may be used as an additional factor in further training or fine-tuning the generative model for generating the synthesized datasets specific to the task of the downstream machine learning model.
    Type: Grant
    Filed: November 15, 2019
    Date of Patent: March 21, 2023
    Assignee: NVIDIA Corporation
    Inventors: Amlan Kar, Aayush Prakash, Ming-Yu Liu, David Jesus Acuna Marrero, Antonio Torralba Barriuso, Sanja Fidler
  • Patent number: 11556797
    Abstract: The present invention relates generally to object annotation, specifically to polygonal annotations of objects. Described are methods of annotating an object including steps of receiving an image depicting an object, generating a set of image features using a CNN encoder implemented on one or more computers, and producing a polygon object annotation via a recurrent decoder or a Graph Neural Network. The recurrent decoder may include a recurrent neural network, a graph neural network or a gated graph neural network. A system for annotating an object and a method of training an object annotation system are also described.
    Type: Grant
    Filed: March 23, 2020
    Date of Patent: January 17, 2023
    Inventors: Sanja Fidler, Amlan Kar, Huan Ling, Jun Gao, Wenzheng Chen, David Jesus Acuna Marrero
  • Publication number: 20220391781
    Abstract: A method performed by a server is provided. The method comprises sending copies of a set of parameters of a hyper network (HN) to at least one client device, receiving from each client device in the at least one client device, a corresponding set of updated parameters of the HN, and determining a next set of parameters of the HN based on the corresponding sets of updated parameters received from the at least one client device. Each client device generates the corresponding set of updated parameters based on a local model architecture of the client device.
    Type: Application
    Filed: May 27, 2022
    Publication date: December 8, 2022
    Inventors: Or Litany, Haggai Maron, David Jesus Acuna Marrero, Jan Kautz, Sanja Fidler, Gal Chechik
  • Publication number: 20220391766
    Abstract: In various examples, systems and methods are disclosed that use a domain-adaptation theory to minimize the reality gap between simulated and real-world domains for training machine learning models. For example, sampling of spatial priors may be used to generate synthetic data that that more closely matches the diversity of data from the real-world. To train models using this synthetic data that still perform well in the real-world, the systems and methods of the present disclosure may use a discriminator that allows a model to learn domain-invariant representations to minimize the divergence between the virtual world and the real-world in a latent space. As such, the techniques described herein allow for a principled approach to learn neural-invariant representations and a theoretically inspired approach on how to sample data from a simulator that, in combination, allow for training of machine learning models using synthetic data.
    Type: Application
    Filed: May 27, 2022
    Publication date: December 8, 2022
    Inventors: David Jesus Acuna Marrero, Sanja Fidler, Jonah Philion
  • Publication number: 20220383073
    Abstract: In various examples, machine learning models (MLMs) may be updated using multi-order gradients in order to train the MLMs, such as at least a first order gradient and any number of higher-order gradients. At least a first of the MLMs may be trained to generate a representation of features that is invariant to a first domain corresponding to a first dataset and a second domain corresponding to a second dataset. At least a second of the MLMs may be trained to classify whether the representation corresponds to the first domain or the second domain. At least a third of the MLMs may trained to perform a task. The first dataset may correspond to a labeled source domain and the second dataset may correspond to an unlabeled target domain. The training may include transferring knowledge from the first domain to the second domain in a representation space.
    Type: Application
    Filed: May 27, 2022
    Publication date: December 1, 2022
    Inventors: David Jesus Acuna Marrero, Sanja Fidler, Marc Law, Guojun Zhang
  • Publication number: 20220044075
    Abstract: Training deep neural networks requires a large amount of labeled training data. Conventionally, labeled training data is generated by gathering real images that are manually labelled which is very time-consuming. Instead of manually labelling a training dataset, domain randomization technique is used generate training data that is automatically labeled. The generated training data may be used to train neural networks for object detection and segmentation (labelling) tasks. In an embodiment, the generated training data includes synthetic input images generated by rendering three-dimensional (3D) objects of interest in a 3D scene. In an embodiment, the generated training data includes synthetic input images generated by rendering 3D objects of interest on a 2D background image. The 3D objects of interest are objects that a neural network is trained to detect and/or label.
    Type: Application
    Filed: October 21, 2021
    Publication date: February 10, 2022
    Inventors: Jonathan Tremblay, Aayush Prakash, Mark A. Brophy, Varun Jampani, Cem Anil, Stanley Thomas Birchfield, Thang Hong To, David Jesus Acuna Marrero
  • Patent number: 11182649
    Abstract: Training deep neural networks requires a large amount of labeled training data. Conventionally, labeled training data is generated by gathering real images that are manually labelled which is very time-consuming. Instead of manually labelling a training dataset, domain randomization technique is used generate training data that is automatically labeled. The generated training data may be used to train neural networks for object detection and segmentation (labelling) tasks. In an embodiment, the generated training data includes synthetic input images generated by rendering three-dimensional (3D) objects of interest in a 3D scene. In an embodiment, the generated training data includes synthetic input images generated by rendering 3D objects of interest on a 2D background image. The 3D objects of interest are objects that a neural network is trained to detect and/or label.
    Type: Grant
    Filed: December 11, 2020
    Date of Patent: November 23, 2021
    Assignee: NVIDIA Corporation
    Inventors: Jonathan Tremblay, Aayush Prakash, Mark A. Brophy, Varun Jampani, Cem Anil, Stanley Thomas Birchfield, Thang Hong To, David Jesus Acuna Marrero
  • Publication number: 20210125077
    Abstract: A computer-implemented method for selecting training data for a neural network, which includes representing a dataset with a mixture of experts model, the mixture of experts model comprising one or more trained neural networks; and generating an application dataset based on one or more performance indicators of one or more of the trained neural networks. Representing the dataset with the mixture of experts model can include partitioning the dataset into one or more data subsets and training one or more neural networks each on one of the data subsets to generate the one or more trained neural networks. A platform for training a neural network and a computer product for carrying out the steps of the method are also described.
    Type: Application
    Filed: September 25, 2020
    Publication date: April 29, 2021
    Inventors: Sanja Fidler, David Jesus Acuna Marrero, Xi Yan
  • Publication number: 20210097346
    Abstract: Training deep neural networks requires a large amount of labeled training data. Conventionally, labeled training data is generated by gathering real images that are manually labelled which is very time-consuming. Instead of manually labelling a training dataset, domain randomization technique is used generate training data that is automatically labeled. The generated training data may be used to train neural networks for object detection and segmentation (labelling) tasks. In an embodiment, the generated training data includes synthetic input images generated by rendering three-dimensional (3D) objects of interest in a 3D scene. In an embodiment, the generated training data includes synthetic input images generated by rendering 3D objects of interest on a 2D background image. The 3D objects of interest are objects that a neural network is trained to detect and/or label.
    Type: Application
    Filed: December 11, 2020
    Publication date: April 1, 2021
    Inventors: Jonathan Tremblay, Aayush Prakash, Mark A. Brophy, Varun Jampani, Cem Anil, Stanley Thomas Birchfield, Thang Hong To, David Jesus Acuna Marrero
  • Patent number: 10867214
    Abstract: Training deep neural networks requires a large amount of labeled training data. Conventionally, labeled training data is generated by gathering real images that are manually labelled which is very time-consuming. Instead of manually labelling a training dataset, domain randomization technique is used generate training data that is automatically labeled. The generated training data may be used to train neural networks for object detection and segmentation (labelling) tasks. In an embodiment, the generated training data includes synthetic input images generated by rendering three-dimensional (3D) objects of interest in a 3D scene. In an embodiment, the generated training data includes synthetic input images generated by rendering 3D objects of interest on a 2D background image. The 3D objects of interest are objects that a neural network is trained to detect and/or label.
    Type: Grant
    Filed: January 24, 2019
    Date of Patent: December 15, 2020
    Assignee: NVIDIA Corporation
    Inventors: Jonathan Tremblay, Aayush Prakash, Mark A. Brophy, Varun Jampani, Cem Anil, Stanley Thomas Birchfield, Thang Hong To, David Jesus Acuna Marrero
  • Publication number: 20200302250
    Abstract: A generative model can be used for generation of spatial layouts and graphs. Such a model can progressively grow these layouts and graphs based on local statistics, where nodes can represent spatial control points of the layout, and edges can represent segments or paths between nodes, such as may correspond to road segments. A generative model can utilize an encoder-decoder architecture where the encoder is a recurrent neural network (RNN) that encodes local incoming paths into a node and the decoder is another RNN that generates outgoing nodes and edges connecting an existing node to the newly generated nodes. Generation is done iteratively, and can finish once all nodes are visited or another end condition is satisfied. Such a model can generate layouts by additionally conditioning on a set of attributes, giving control to a user in generating the layout.
    Type: Application
    Filed: March 20, 2020
    Publication date: September 24, 2020
    Inventors: Hang Chu, Daiqing Li, David Jesus Acuna Marrero, Amlan Kar, Maria Shugrina, Ming-Yu Liu, Antonio Torralba Barriuso, Sanja Fidler