Abstract: Apparatuses, system, and techniques to process resources used to perform a neural network to generate one or more images. In at least one embodiment, a processor comprising circuitry uses one or more neural networks to generate one or more second images based, at least in part, on noise within a first image.

Abstract: A world foundation model (WFM) is a digital twin of the world and is used for simulating a physical AI system including sensors and actuators. In an embodiment, the architecture of the WFM comprises a diffusion model based on a transformer architecture. The diffusion model is first trained for text to video generation to map text prompts to videos of visual worlds. The diffusion model is then extended to accept video input (current observation) in addition to the text prompt (perturbation) to generate output video corresponding to future observations based on the video frames and text prompt. The WFM generates the output video while maintaining three-dimensional consistency and physics accuracy between the input video frames and each successive frame in the output video. The resulting diffusion-based WFM is general-purpose world model that can be fine-tuned to create a customized world model.

Abstract: World foundation models (WFMs) are trained to process video frames (observations) and a text prompt (perturbation) to generate output video corresponding to future observations based on the video frames and text prompt. Training WFMs requires a large amount of high-quality video training data with diverse content and action that is consistent with the physical world. The WFMs are trained to generate output video while maintaining three-dimensional consistency and physics accuracy. An image data curation pipeline is implemented that may be scaled to process large quantities of video data to produce a high-quality video training dataset.

Abstract: The disclosed method for generating images includes performing, based on one or more inputs, one or more first denoising diffusion operations using a first trained machine learning model to generate a first image at a first resolution; and performing, based on the one or more inputs and the first image, one or more second denoising diffusion operations using a second trained machine learning model to generate a second image at a second resolution.

Abstract: The disclosed method for generating images includes performing, based on one or more inputs, one or more first denoising diffusion operations using a first trained machine learning model to generate a first image at a first resolution; and performing, based on the one or more inputs and the first image, one or more second denoising diffusion operations using a second trained machine learning model to generate a second image at a second resolution.

Abstract: Techniques are disclosed herein for generating a content item. The techniques include performing one or more first denoising operations based on an input and a first machine learning model to generate a first content item, and performing one or more second denoising operations based on the input, the first content item, and a second machine learning model to generate a second content item, where the first machine learning model is trained to denoise content items having an amount of corruption within a first corruption range, the second machine learning model is trained to denoise content items having an amount of corruption within a second corruption range, and the second corruption range is lower than the first corruption range.