Abstract: The disclosed method for generating molecules includes selecting, based on one or more molecule properties, one or more hard molecule fragments and one or more soft molecule fragments; and processing, using a trained machine learning model, the one or more hard molecule fragments and the one or more soft molecule fragments to generate a molecule, where the molecule includes the one or more hard molecule fragments, and the trained machine learning model generates the molecule based on the one or more soft molecule fragments.

Abstract: Generalizable feature distillation systems that align 3D features with 2D foundation model features using a feedforward network, avoiding per-scene optimization, and a flexible end-to-end 3D scene interpretation system that applies the extracted 3D features and pretrained 2D vision-language models for various 3D scene understanding tasks.

Abstract: A machine learning model (MLM) may be trained and evaluated. Attribute-based performance metrics may be analyzed to identify attributes for which the MLM is performing below a threshold when each are present in a sample. A generative neural network (GNN) may be used to generate samples including compositions of the attributes, and the samples may be used to augment the data used to train the MLM. This may be repeated until one or more criteria are satisfied. In various examples, a temporal sequence of data items, such as frames of a video, may be generated which may form samples of the data set. Sets of attribute values may be determined based on one or more temporal scenarios to be represented in the data set, and one or more GNNs may be used to generate the sequence to depict information corresponding to the attribute values.

Abstract: One embodiment sets forth a technique for creating a generative model. The technique includes generating a trained generative model with a first component that converts data points in the training dataset into latent variable values, a second component that learns a distribution of the latent variable values, and a third component that converts the latent variable values into output distributions. The technique also includes training an energy-based model to learn an energy function based on values sampled from a first distribution associated with the training dataset and values sampled from a second distribution during operation of the trained generative model. The technique further includes creating a joint model that includes one or more portions of the trained generative model and the energy-based model, and that applies energy values from the energy-based model to samples from the second distribution to produce additional values used to generate a new data point.

Abstract: Systems and methods are disclosed that generate blob video representations such as blob video parameters and blob video descriptions and use the blob video representations to generate videos. For example, embodiments of the present disclosure may decompose videos into visual primitives (e.g., blob video representations, which may be general representations for controllable video generation). Based on the blob video representations, a blob-grounded text-to-video diffusion model that includes masked three-dimensional (3D) self-attention layers and/or masked spatial cross-attention layers may be developed. The masked 3D self-attention layers and/or masked spatial cross-attention layers may effectively improve regional consistency across frames. Additionally, and/or alternatively, embodiments of the present disclosure may utilize context interpolation that may interpolate text embeddings.

Abstract: Apparatuses, systems, and techniques for probabilistic forecasting. In at least one embodiment, a neural network is configured to receive an input frame indicating a current status, and obtain, based on the input, a sequence of first frames to form a first processing window. The first frames in the sequence are corrupted with one or more predefined noise schedules. The neural network is configured to further denoise the sequence of first frames within the first processing window simultaneously to produce a sequence of second frames, output one or more second frames from the first processing window, and form a second processing window by appending one or more additional frames to the remaining one or more second frames from the first processing window. As such, the neural network produces output frames based on a rolling processing window.

Abstract: Energy-based diffusion models for transforming noisy inputs, the energy-based diffusion models including a denoiser model configured to transform a noisy input into a multiple candidate output predictions at each of a plurality of denoising iterations, and an energy-based model configured to transform the candidate output predictions at each denoising iteration into a single output prediction.

Abstract: De novo protein design, the rational design of new proteins from scratch with specific functions and properties, is a grand challenge in molecular biology. Recently, deep generative models have emerged as a novel data-driven tool for protein engineering. However, current diffusion- and flow-based models generally synthesize backbones only, without sequence or side chains, while protein language models often model sequences instead. The present disclosure provides flow-based protein structure generation which can be conditioned on a given fold class.

Abstract: Systems and methods are disclosed that perform training of a flow-based generative model for three-dimensional (3D) point cloud generation. For example, the method may include obtaining offline optimal transport (OT) maps for a training set comprising 3D point clouds. The method further includes randomly sampling from the training set to obtain data samples indicating points from 3D point clouds and determining corresponding noise samples associated with the data samples based on the offline OT maps. The method also includes obtaining modified noise samples based on adding noise to perturb the corresponding noise samples and training the flow-based generative model based on the modified noise samples and the data samples.

Abstract: Diffusion models are machine learning algorithms implemented as neural network-based denoisers that are uniquely trained to generate high-quality data from an input lower-quality data. However, for complex datasets, the samples generated by a diffusion model can still fail to reproduce the quality and diversity of the training data, due to approximation errors made by the finite-capacity network. Diffusion guidance addresses this issue by steering the sampling process away from a less desired model, toward a preferred one. However, current guidance methods rely on a single additional denoiser and manual tuning of guidance weights, which is suboptimal for large, complex models and leads to inefficiencies. The present disclosure employs a mixture of denoisers to guide a diffusion model, which can increase the expressiveness of the guidance and substantially improve sample quality and diversity in the output of the diffusion model.

Abstract: Approaches presented herein provide for the generation of synthesized data from input noise using a denoising diffusion network. A higher order differential equation solver can be used for the denoising process, with one or more higher-order terms being distilled into one or more separate efficient neural networks. A separate, efficient neural network can be called together with a primary denoising model at inference time without significant loss in sampling efficiency. The separate neural network can provide information about the curvature (or other higher-order term) of the differential equation, representing a denoising trajectory, that can be used by the primary diffusion network to denoise the image using fewer denoising iterations.

Abstract: Systems and methods are disclosed that perform a truncated consistency model training framework that includes two stages. For example, in the first stage, embodiments of the present disclosure may train a consistency model using first and second time step samples. The first time step samples may be obtained based on sampling from a plurality of time steps. Following, a truncated time range that does not include all of the time steps from the plurality of time steps is obtained. Then, third time step samples are obtained based on sampling from the truncated time range and fourth time step samples are determined based on the third time step samples and a time step difference. Afterwards, in a second stage, the consistency model is further trained using the third time step samples and the fourth time step samples.

Abstract: An open-vocabulary diffusion-based panoptic segmentation system is not limited to perform segmentation using only object categories seen during training, and instead can also successfully perform segmentation of object categories not seen during training and only seen during testing and inferencing. In contrast with conventional techniques, a text-conditioned diffusion (generative) model is used to perform the segmentation. The text-conditioned diffusion model is pre-trained to generate images from text captions, including computing internal representations that provide spatially well-differentiated object features. The internal representations computed within the diffusion model comprise object masks and a semantic visual representation of the object. The semantic visual representation may be extracted from the diffusion model and used in conjunction with a text representation of a category label to classify the object.

Abstract: Apparatuses, systems, and techniques for adaptive flow matching. In at least one embodiment, input is received, which includes one or more first variables at first scale. An encoder is used to encode the input to provide a base distribution at the first scale. The base distribution is associated with one or more second variables, and the one or more second variables include one or more variables absent from the one or more first variables. A perturbed base distribution is obtained based on the base distribution and an adaptive noise. A diffusion model is used to generate a target distribution at a second scale. The target distribution is associated with the one or more second variables. The second scale is finer than the first scale.

Abstract: A generative framework enables transformation of a conventional Gaussian diffusion model for modeling heavy-tailed distributions, such as the data distributions typical of scientific applications. In an embodiment, the denoising model predicts short-term or long-term events based on input data (e.g., certain weather or financial variables). In an embodiment, the denoising model generates high resolution data, such as generating local weather forecasts or conditions from certain weather variables for a larger region.

Abstract: The disclosed method for generating proteins includes generating, using a trained machine learning model, a first protein based on a three-dimensional (3D) representation of a spatial layout for the first protein, where generating the first protein comprises applying cross-attention between one or more first tokens associated with the 3D representation and one or more second tokens associated with a second protein.

Abstract: Diffusion models trained on largescale internet datasets have demonstrated an exceptional ability to generate high-quality and photorealistic two-dimensional (2D) images across diverse styles and domains. Generating three-dimensional (3D) scenes, however, is much more challenging and much less explored due to the lack of training data and the presence of many objects that necessitates compositionality and consistency across different views and objects. The present disclosure uses 3D blobs to create a compositional 3D scene representation from which 2D views can be generated.

Abstract: Approaches presented herein provide for the generation of synthesized data from input noise using a denoising diffusion network. A higher order differential equation solver can be used for the denoising process, with one or more higher-order terms being distilled into one or more separate efficient neural networks. A separate, efficient neural network can be called together with a primary denoising model at inference time without significant loss in sampling efficiency. The separate neural network can provide information about the curvature (or other higher-order term) of the differential equation, representing a denoising trajectory, that can be used by the primary diffusion network to denoise the image using fewer denoising iterations.

Abstract: Approaches presented herein provide for the unconditional generation of novel three dimensional (3D) object shape representations, such as point clouds or meshes. In at least one embodiment, a first denoising diffusion model (DDM) can be trained to synthesize a 1D shape latent from Gaussian noise, and a second DDM can be trained to generate a set of latent points conditioned on this 1D shape latent. The shape latent and set of latent points can be provided to a decoder to generate a 3D point cloud representative of a random object from among the object classes on which the models were trained. A surface reconstruction process may be used to generate a surface mesh from this generated point cloud. Such an approach can scale to complex and/or multimodal distributions, and can be highly flexible as it can be adapted to various tasks such as multimodal voxel- or text-guided synthesis.

Abstract: In some aspects, the present disclosure provides a method for generating a geometrical structure of a binding complex formed between a protein and a ligand. In some embodiments, the method comprises sampling an initial geometrical structure of the binding complex from a geometry prior. In some embodiments, the method comprises denoising, using a machine-learned stochastic differential equation (SDE), the initial geometrical structure to generate the geometrical structure of the binding complex.