Abstract: The disclosed method for generating output climate data includes receiving a user control input, input climate data, input conditioning data, and input mask data, generating, based on the user control input, an inference strategy, and performing, using at least one of a first diffusion model, a second diffusion model, or a third diffusion model, one or more reverse diffusion operations to generate the output climate data based on the inference strategy, the input climate data, the input conditioning data, and the input mask data.

Abstract: The disclosed method for training a diffusion model includes generating, based on conditioning data, one or more conditioning embeddings, generating, based on climate data and mask data, masked climate data, adding noise to the masked climate data to generate noisy masked climate data, performing, using the diffusion model, a denoising step to generate predicted masked climate data based on the one or more conditioning embeddings and the noisy masked climate data, calculating a loss based on the mask data, the masked climate data, and the predicted masked climate data, and updating, based on the loss, one or more parameters of the diffusion model.

Abstract: The disclosed method for training a diffusion model includes generating, based on conditioning data, one or more conditioning embeddings, generating, based on climate data and mask data, temporal masked climate data, a temporal mask, and one or more channel and temporal masks, adding noise to the temporal masked climate data to generate noisy temporal masked climate data, performing, using the diffusion model, a denoising operation to generate predicted temporal masked climate data based on the one or more conditioning embeddings, the noisy temporal masked climate data, and the temporal mask, calculating a loss based on the one or more channel and temporal masks, the temporal masked climate data, and the predicted temporal masked climate data, and updating, based on the loss, one or more parameters of the diffusion model.

Abstract: Two and three dimensional video diffusion models configured via training to be equivariant to spatial transformations of the input by training the model using warped noise, where equivariance of the models is learned using a conventional video diffusion loss function modified to replace independent noise inputs with warped noise inputs.

Abstract: Systems and methods are disclosed that improve performance of synthesized motion generated by a diffusion neural network model. A physics-guided motion diffusion model incorporates physical constraints into the diffusion process to model the complex dynamics induced by forces and contact. Specifically, a physics-based motion projection module uses motion imitation in a physics simulator to project the denoised motion of a diffusion step to a physically plausible motion. The projected motion is further used in the next diffusion iteration to guide the denoising diffusion process. The use of physical constraints in the physics-guided motion diffusion model iteratively pulls the motion toward a physically-plausible space, reducing artifacts such as floating, foot sliding, and ground penetration.

Abstract: A machine learning model (MLM) may be trained and evaluated. Attribute-based performance metrics may be analyzed to identify attributes for which the MLM is performing below a threshold when each are present in a sample. A generative neural network (GNN) may be used to generate samples including compositions of the attributes, and the samples may be used to augment the data used to train the MLM. This may be repeated until one or more criteria are satisfied. In various examples, a temporal sequence of data items, such as frames of a video, may be generated which may form samples of the data set. Sets of attribute values may be determined based on one or more temporal scenarios to be represented in the data set, and one or more GNNs may be used to generate the sequence to depict information corresponding to the attribute values.

Abstract: One embodiment of the present invention sets forth a technique for performing machine learning. The technique includes inputting a training dataset into a variational autoencoder (VAE) comprising an encoder network, a prior network, and a decoder network. The technique also includes training the VAE by updating one or more parameters of the VAE based on a smoothness of one or more outputs produced by the VAE from the training dataset. The technique further includes producing generative output that reflects a first distribution of the training dataset by applying the decoder network to one or more values sampled from a second distribution of latent variables generated by the prior network.

Abstract: In various examples, systems and methods are disclosed relating to neural networks for synthetic data generation with discrete and continuous variable features. In training, an encoder can determine a plurality of encodings from a plurality of samples of training data, and the continuous generative model can operate as a decoder that is conditioned on the plurality of encodings to generate an estimated output to update the encoder and the continuous generative model. The discrete generative model can be trained over the plurality of encodings to learn to generate discrete variables corresponding to the distribution of information represented by the training data. At runtime, the discrete generative model can be used to generate a discrete variable from an input prompt, and can provide the discrete variable to the continuous generative model for the continuous generative model to generate an output, such an image, conditioned on the discrete variable.

Abstract: ƒ-distill is a framework for distributional matching distillation based on ƒ-divergence minimization. A student diffusion model (DM) can be trained by generating an output content, scoring a student distribution of data values, and then comparing the student scores to a teacher DM scoring. The scoring process can use a gradient algorithm that scores multiple regions of the generated content individually. The gradient algorithm can comprise the product of the weighting function and the score difference between the teacher and student distributions across each region of the generated content. Each region's score can be weighted separately from other regions based on data distributions, mode-seeking parameters, or other factors. ƒ-distill can encompass variational score distillation objectives while allowing less mode-seeking divergences. In some aspects, by leveraging the weighting function, ƒ-distill can down weight regions with larger score estimation errors.

Abstract: Designing binding proteins is a central challenge in computational biology. Modern artificial intelligence (AI)-based approaches fall into two classes: (1) generative methods which treat binder design as conditional generation, training on binder-target complex structures and producing new candidates for unseen targets; and (2) hallucination methods which use the confidence and alignment scores of structure predictors to assess interfaces and optimize binder sequences via gradient feedback. However, these current approaches do not unify the generative and hallucination methods. The present disclosure provides synthetic protein binder generation conditioned on a given target protein via a framework that unifies a flow-based base generative model with flexible inference-time optimization utilizing the generative prior, thereby combining the strengths of both approaches.

Abstract: Candidate drug exploration processes utilizing a masked discrete diffusion model with fragment remasking to generate molecular sequences, wherein the operation of the masked discrete diffusion model is conditioned with molecular context guidance, and fragment remasking involves generating molecular fragment candidates, masking the molecular fragment candidates, scoring the molecular fragment candidates, and attaching the molecular fragment candidates to a molecule.

Abstract: The disclosed method for generating molecules includes selecting, based on one or more molecule properties, one or more hard molecule fragments and one or more soft molecule fragments; and processing, using a trained machine learning model, the one or more hard molecule fragments and the one or more soft molecule fragments to generate a molecule, where the molecule includes the one or more hard molecule fragments, and the trained machine learning model generates the molecule based on the one or more soft molecule fragments.

Abstract: Generalizable feature distillation systems that align 3D features with 2D foundation model features using a feedforward network, avoiding per-scene optimization, and a flexible end-to-end 3D scene interpretation system that applies the extracted 3D features and pretrained 2D vision-language models for various 3D scene understanding tasks.

Abstract: A machine learning model (MLM) may be trained and evaluated. Attribute-based performance metrics may be analyzed to identify attributes for which the MLM is performing below a threshold when each are present in a sample. A generative neural network (GNN) may be used to generate samples including compositions of the attributes, and the samples may be used to augment the data used to train the MLM. This may be repeated until one or more criteria are satisfied. In various examples, a temporal sequence of data items, such as frames of a video, may be generated which may form samples of the data set. Sets of attribute values may be determined based on one or more temporal scenarios to be represented in the data set, and one or more GNNs may be used to generate the sequence to depict information corresponding to the attribute values.

Abstract: One embodiment sets forth a technique for creating a generative model. The technique includes generating a trained generative model with a first component that converts data points in the training dataset into latent variable values, a second component that learns a distribution of the latent variable values, and a third component that converts the latent variable values into output distributions. The technique also includes training an energy-based model to learn an energy function based on values sampled from a first distribution associated with the training dataset and values sampled from a second distribution during operation of the trained generative model. The technique further includes creating a joint model that includes one or more portions of the trained generative model and the energy-based model, and that applies energy values from the energy-based model to samples from the second distribution to produce additional values used to generate a new data point.

Abstract: Systems and methods are disclosed that generate blob video representations such as blob video parameters and blob video descriptions and use the blob video representations to generate videos. For example, embodiments of the present disclosure may decompose videos into visual primitives (e.g., blob video representations, which may be general representations for controllable video generation). Based on the blob video representations, a blob-grounded text-to-video diffusion model that includes masked three-dimensional (3D) self-attention layers and/or masked spatial cross-attention layers may be developed. The masked 3D self-attention layers and/or masked spatial cross-attention layers may effectively improve regional consistency across frames. Additionally, and/or alternatively, embodiments of the present disclosure may utilize context interpolation that may interpolate text embeddings.

Abstract: Apparatuses, systems, and techniques for probabilistic forecasting. In at least one embodiment, a neural network is configured to receive an input frame indicating a current status, and obtain, based on the input, a sequence of first frames to form a first processing window. The first frames in the sequence are corrupted with one or more predefined noise schedules. The neural network is configured to further denoise the sequence of first frames within the first processing window simultaneously to produce a sequence of second frames, output one or more second frames from the first processing window, and form a second processing window by appending one or more additional frames to the remaining one or more second frames from the first processing window. As such, the neural network produces output frames based on a rolling processing window.

Abstract: Energy-based diffusion models for transforming noisy inputs, the energy-based diffusion models including a denoiser model configured to transform a noisy input into a multiple candidate output predictions at each of a plurality of denoising iterations, and an energy-based model configured to transform the candidate output predictions at each denoising iteration into a single output prediction.

Abstract: De novo protein design, the rational design of new proteins from scratch with specific functions and properties, is a grand challenge in molecular biology. Recently, deep generative models have emerged as a novel data-driven tool for protein engineering. However, current diffusion- and flow-based models generally synthesize backbones only, without sequence or side chains, while protein language models often model sequences instead. The present disclosure provides flow-based protein structure generation which can be conditioned on a given fold class.

Abstract: Systems and methods are disclosed that perform training of a flow-based generative model for three-dimensional (3D) point cloud generation. For example, the method may include obtaining offline optimal transport (OT) maps for a training set comprising 3D point clouds. The method further includes randomly sampling from the training set to obtain data samples indicating points from 3D point clouds and determining corresponding noise samples associated with the data samples based on the offline OT maps. The method also includes obtaining modified noise samples based on adding noise to perturb the corresponding noise samples and training the flow-based generative model based on the modified noise samples and the data samples.