Patents by Inventor Arash Vahdat
Arash Vahdat has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20240169636Abstract: Systems and methods are disclosed that improve performance of synthesized motion generated by a diffusion neural network model. A physics-guided motion diffusion model incorporates physical constraints into the diffusion process to model the complex dynamics induced by forces and contact. Specifically, a physics-based motion projection module uses motion imitation in a physics simulator to project the denoised motion of a diffusion step to a physically plausible motion. The projected motion is further used in the next diffusion iteration to guide the denoising diffusion process. The use of physical constraints in the physics-guided motion diffusion model iteratively pulls the motion toward a physically-plausible space, reducing artifacts such as floating, foot sliding, and ground penetration.Type: ApplicationFiled: May 15, 2023Publication date: May 23, 2024Inventors: Ye Yuan, Jiaming Song, Umar Iqbal, Arash Vahdat, Jan Kautz
-
Publication number: 20240161250Abstract: Techniques are disclosed herein for generating a content item. The techniques include performing one or more first denoising operations based on an input and a first machine learning model to generate a first content item, and performing one or more second denoising operations based on the input, the first content item, and a second machine learning model to generate a second content item, where the first machine learning model is trained to denoise content items having an amount of corruption within a first corruption range, the second machine learning model is trained to denoise content items having an amount of corruption within a second corruption range, and the second corruption range is lower than the first corruption range.Type: ApplicationFiled: October 11, 2023Publication date: May 16, 2024Inventors: Yogesh BALAJI, Timo Oskari AILA, Miika AITTALA, Bryan CATANZARO, Xun HUANG, Tero Tapani KARRAS, Karsten KREIS, Samuli LAINE, Ming-Yu LIU, Seungjun NAH, Jiaming SONG, Arash VAHDAT, Qinsheng ZHANG
-
Publication number: 20240153093Abstract: An open-vocabulary diffusion-based panoptic segmentation system is not limited to perform segmentation using only object categories seen during training, and instead can also successfully perform segmentation of object categories not seen during training and only seen during testing and inferencing. In contrast with conventional techniques, a text-conditioned diffusion (generative) model is used to perform the segmentation. The text-conditioned diffusion model is pre-trained to generate images from text captions, including computing internal representations that provide spatially well-differentiated object features. The internal representations computed within the diffusion model comprise object masks and a semantic visual representation of the object. The semantic visual representation may be extracted from the diffusion model and used in conjunction with a text representation of a category label to classify the object.Type: ApplicationFiled: May 1, 2023Publication date: May 9, 2024Inventors: Jiarui Xu, Shalini De Mello, Sifei Liu, Arash Vahdat, Wonmin Byeon
-
Patent number: 11978258Abstract: Apparatuses, systems, and techniques to identify out-of-distribution input data in one or more neural networks. In at least one embodiment, a technique includes training one or more neural networks to infer a plurality of characteristics about input information based, at least in part, on the one or more neural networks being independently trained to infer each of the plurality of characteristics about the input information.Type: GrantFiled: April 6, 2021Date of Patent: May 7, 2024Assignee: NVIDIA CorporationInventors: Sina Mohseni, Arash Vahdat, Jay Yadawa
-
Publication number: 20240144000Abstract: A neural network model is trained for fairness and accuracy using both real and synthesized training data, such as images. During training a first sampling ratio between the real and synthesized training data is optimized. The first sampling ratio may comprise a value for each group (or attribute), where each value is optimized. A second sampling ratio defines relative amounts of training data that are used for each one of the groups. Furthermore, a neural network model accuracy and a fairness metric are both used for updating the first and second sampling ratios during training iterations. The neural network model may be trained using different classes of training data. The second sampling ratio may vary for each class.Type: ApplicationFiled: April 26, 2023Publication date: May 2, 2024Inventors: Yuji Roh, Weili Nie, De-An Huang, Arash Vahdat, Animashree Anandkumar
-
Publication number: 20240111894Abstract: In various examples, systems and methods are disclosed relating to differentially private generative machine learning models. Systems and methods are disclosed for configuring generative models using privacy criteria, such as differential privacy criteria. The systems and methods can generate outputs representing content using machine learning models, such as diffusion models, that are determined in ways that satisfy differential privacy criteria. The machine learning models can be determined by diffusing the same training data to multiple noise levels.Type: ApplicationFiled: February 3, 2023Publication date: April 4, 2024Applicant: NVIDIA CorporationInventors: Karsten Julian KREIS, Tim DOCKHORN, Tianshi CAO, Arash VAHDAT
-
Patent number: 11948078Abstract: The disclosure provides a framework or system for learning visual representation using a large set of image/text pairs. The disclosure provides, for example, a method of visual representation learning, a joint representation learning system, and an artificial intelligence (AI) system that employs one or more of the trained models from the method or system. The AI system can be used, for example, in autonomous or semi-autonomous vehicles. In one example, the method of visual representation learning includes: (1) receiving a set of image embeddings from an image representation model and a set of text embeddings from a text representation model, and (2) training, employing mutual information, a critic function by learning relationships between the set of image embeddings and the set of text embeddings.Type: GrantFiled: August 21, 2020Date of Patent: April 2, 2024Assignee: NVIDIA CorporationInventors: Arash Vahdat, Tanmay Gupta, Xiaodong Yang, Jan Kautz
-
Publication number: 20240104698Abstract: Apparatuses, systems, and techniques are presented to remove unintended variations introduced into data. In at least one embodiment, a first image of an object can be generated based, at least in part, upon adding noise to, and removing the noise from, a second image of the object.Type: ApplicationFiled: April 12, 2022Publication date: March 28, 2024Inventors: Weili Nie, Yujia Huang, Chaowei Xiao, Arash Vahdat, Anima Anandkumar
-
Publication number: 20240096115Abstract: Landmark detection refers to the detection of landmarks within an image or a video, and is used in many computer vision tasks such emotion recognition, face identity verification, hand tracking, gesture recognition, and eye gaze tracking. Current landmark detection methods rely on a cascaded computation through cascaded networks or an ensemble of multiple models, which starts with an initial guess of the landmarks and iteratively produces corrected landmarks which match the input more finely. However, the iterations required by current methods typically increase the training memory cost linearly, and do not have an obvious stopping criteria. Moreover, these methods tend to exhibit jitter in landmark detection results for video. The present disclosure improves current landmark detection methods by providing landmark detection using an iterative neural network.Type: ApplicationFiled: September 7, 2023Publication date: March 21, 2024Inventors: Pavlo Molchanov, Jan Kautz, Arash Vahdat, Hongxu Yin, Paul Micaelli
-
Publication number: 20240005604Abstract: Approaches presented herein provide for the unconditional generation of novel three dimensional (3D) object shape representations, such as point clouds or meshes. In at least one embodiment, a first denoising diffusion model (DDM) can be trained to synthesize a 1D shape latent from Gaussian noise, and a second DDM can be trained to generate a set of latent points conditioned on this 1D shape latent. The shape latent and set of latent points can be provided to a decoder to generate a 3D point cloud representative of a random object from among the object classes on which the models were trained. A surface reconstruction process may be used to generate a surface mesh from this generated point cloud. Such an approach can scale to complex and/or multimodal distributions, and can be highly flexible as it can be adapted to various tasks such as multimodal voxel- or text-guided synthesis.Type: ApplicationFiled: May 19, 2023Publication date: January 4, 2024Inventors: Karsten Julian Kreis, Xiaohui Zeng, Arash Vahdat, Francis Williams, Zan Gojcic, Or Litany, Sanja Fidler
-
Patent number: 11847538Abstract: Apparatuses, systems, and techniques to train a generative model based at least in part on a private dataset. In at least one embodiment, the generative model is trained based at least in part on a differentially private Sinkhorn algorithm, for example, using backpropagation with gradient descent to determine a gradient of a set of parameters of the generative models and modifying the set of parameters based at least in part on the gradient.Type: GrantFiled: May 11, 2021Date of Patent: December 19, 2023Assignee: NVIDIA CorporationInventors: Tianshi Cao, Alex Bie, Karsten Julian Kreis, Sanja Fidler, Arash Vahdat
-
Publication number: 20230377099Abstract: Approaches presented herein provide for the generation of synthesized data from input noise using a denoising diffusion network. A higher order differential equation solver can be used for the denoising process, with one or more higher-order terms being distilled into one or more separate efficient neural networks. A separate, efficient neural network can be called together with a primary denoising model at inference time without significant loss in sampling efficiency. The separate neural network can provide information about the curvature (or other higher-order term) of the differential equation, representing a denoising trajectory, that can be used by the primary diffusion network to denoise the image using fewer denoising iterations.Type: ApplicationFiled: May 18, 2023Publication date: November 23, 2023Inventors: Karsten Julian Kreis, Tim Dockhorn, Arash Vahdat
-
Publication number: 20230351807Abstract: A machine learning model (MLM) may be trained and evaluated. Attribute-based performance metrics may be analyzed to identify attributes for which the MLM is performing below a threshold when each are present in a sample. A generative neural network (GNN) may be used to generate samples including compositions of the attributes, and the samples may be used to augment the data used to train the MLM. This may be repeated until one or more criteria are satisfied. In various examples, a temporal sequence of data items, such as frames of a video, may be generated which may form samples of the data set. Sets of attribute values may be determined based on one or more temporal scenarios to be represented in the data set, and one or more GNNs may be used to generate the sequence to depict information corresponding to the attribute values.Type: ApplicationFiled: May 2, 2022Publication date: November 2, 2023Inventors: Yuzhuo Ren, Weili Nie, Arash Vahdat, Animashree Anandkumar, Nishant Puri, Niranjan Avadhanam
-
Publication number: 20230186077Abstract: One embodiment of the present invention sets forth a technique for executing a transformer neural network. The technique includes computing a first set of halting scores for a first set of tokens that has been input into a first layer of the transformer neural network. The technique also includes determining that a first halting score included in the first set of halting scores exceeds a threshold value. The technique further includes in response to the first halting score exceeding the threshold value, causing a first token that is included in the first set of tokens and is associated with the first halting score not to be processed by one or more layers within the transformer neural network that are subsequent to the first layer.Type: ApplicationFiled: June 15, 2022Publication date: June 15, 2023Inventors: Hongxu YIN, Jan KAUTZ, Jose Manuel ALVAREZ LOPEZ, Arun MALLYA, Pavlo MOLCHANOV, Arash VAHDAT
-
Patent number: 11625612Abstract: The domain adaptation problem is addressed by using the predictions of a trained model over both source and target domain to retain the model with the assistance of an auxiliary model and a modified objective function. Inaccuracy in the model's predictions in the target domain is treated as noise and is reduced by using a robust learning framework during retraining, enabling unsupervised training in the target domain. Applications include object detection models, where noise in retraining is reduced by explicitly representing label noise and geometry noise in the objective function and using the ancillary model to inject information about label noise.Type: GrantFiled: January 31, 2020Date of Patent: April 11, 2023Assignee: D-WAVE SYSTEMS INC.Inventors: Arash Vahdat, Mani Ranjbar, Mehran Khodabandeh, William G. Macready, Zhengbing Bian
-
Publication number: 20230109379Abstract: Systems and methods described relate to the synthesis of content using generative models. In at least one embodiment, a score-based generative model can use a stochastic differential equation with critically-damped Langevin diffusion to learn to synthesize content. During a forward diffusion process, noise can be introduced into a set of auxiliary (e.g., “velocity”) values for an input image to learn a score function. This score function can be used with the stochastic differential equation during a reverse diffusion denoising process to remove noise from the image to generate a reconstructed version of the input image. A score matching objective for the critically-damped Langevin diffusion process can require only the conditional distribution learned from the velocity data. A stochastic differential equation-based integrator can then allow for efficient sampling from these critically-damped Langevin diffusion-based models.Type: ApplicationFiled: October 4, 2022Publication date: April 6, 2023Inventors: Karsten Kreis, Tim Dockhorn, Arash Vahdat
-
Publication number: 20230095092Abstract: Apparatuses, systems, and techniques are presented to train and utilize one or more neural networks. A denoising diffusion generative adversarial network (denoising diffusion GAN) reduces a number of denoising steps during a reverse process. The denoising diffusion GAN does not assume a Gaussian distribution for large steps of the denoising process and applies a multi-model model to permit denoising with fewer steps. Systems and methods further minimize a divergence between a diffused real data distribution and a diffused generator distribution over several timesteps. Accordingly, various embodiments may enable faster sample generation, in which the samples are generated from noise using the denoising diffusion GAN.Type: ApplicationFiled: September 30, 2022Publication date: March 30, 2023Inventors: Zhisheng Xiao, Karsten Kreis, Arash Vahdat
-
Publication number: 20230015253Abstract: Apparatuses, systems, and techniques are presented to generate one or more images comprising one or more objects based, at least in part, on one or more dynamically configurable attributes of the one or objects. In at least one embodiment, one or more images comprising one or more objects can be generated based, at least in part, on one or more dynamically configurable attributes of the one or objects.Type: ApplicationFiled: October 19, 2021Publication date: January 19, 2023Inventors: Weili Nie, Arash Vahdat, Anima Anandkumar
-
Publication number: 20220405583Abstract: One embodiment of the present invention sets forth a technique for training a generative model. The technique includes converting a first data point included in a training dataset into a first set of values associated with a base distribution for a score-based generative model. The technique also includes performing one or more denoising operations via the score-based generative model to convert the first set of values into a first set of latent variable values associated with a latent space. The technique further includes performing one or more additional operations to convert the first set of latent variable values into a second data point. Finally, the technique includes computing one or more losses based on the first data point and the second data point and generating a trained generative model based on the one or more losses, wherein the trained generative model includes the score-based generative model.Type: ApplicationFiled: February 25, 2022Publication date: December 22, 2022Inventors: Arash VAHDAT, Karsten KREIS, Jan KAUTZ
-
Patent number: 11531852Abstract: Machine learning classification models which are robust against label noise are provided. Noise may be modelled explicitly by modelling “label flips”, where incorrect binary labels are “flipped” relative to their ground truth value. Distributions of label flips may be modelled as prior and posterior distributions in a flexible architecture for machine learning systems. An arbitrary classification model may be provided within the system. The classification model is made more robust to label noise by operation of the prior and posterior distributions. Particular prior and approximating posterior distributions are disclosed.Type: GrantFiled: November 27, 2017Date of Patent: December 20, 2022Assignee: D-WAVE SYSTEMS INC.Inventor: Arash Vahdat