Abstract: Neural network architectures and machine learning techniques that support tokenization of raw visual input to generate a compact representation in a latent feature space as well as de-tokenization to generate raw visual output. In at least one embodiment, tokenization systems and methods leverages wavelet transforms and causal operations to capture spatial and temporal dependencies in the raw visual input.

Abstract: The disclosed method for generating images includes performing, based on one or more inputs, one or more first denoising diffusion operations using a first trained machine learning model to generate a first image at a first resolution; and performing, based on the one or more inputs and the first image, one or more second denoising diffusion operations using a second trained machine learning model to generate a second image at a second resolution.

Abstract: The disclosed method for generating images includes performing, based on one or more inputs, one or more first denoising diffusion operations using a first trained machine learning model to generate a first image at a first resolution; and performing, based on the one or more inputs and the first image, one or more second denoising diffusion operations using a second trained machine learning model to generate a second image at a second resolution.

Abstract: The disclosed method for quantizing one or more latent embeddings includes receiving one or more latent embeddings, generating, based on the one or more latent embeddings, one or more channel groups, and generating, based on the one or more channel groups, one or more quantized latent embeddings.

Abstract: The disclosed method for tokenizing video frames includes receiving one or more video frames from one or more I/O devices, generating, using an encoder, one or more latent embeddings based on the one or more video frames, wherein the encoder comprises one or more patchify modules, one or more spatial-temporal Mamba modules, and one or more token pooling modules, and generating, using a quantizer, one or more quantized latent embeddings based on the one or more latent embeddings.

Abstract: Apparatuses, systems, and techniques to generate interpolated video frames. In at least one embodiment, an interpolated video frame is generated based, at least in part, on a first set of pixel data sampled from a first video frame, and a second set of pixel data sampled from a second video frame based, at least in part, on a set of forward pointing motion vectors from the first video frame to the second video frame.

Abstract: Video compression systems based on a variational autoencoder, the variational autoencoder including an encoder and a decoder coupled via a latent space embedding component, the encoder configured to transform an input video into a feature maps of the input video at different feature resolution scales, the latent space embedding component configured to transform the feature maps into a latent space parameter distribution, and the decoder configured to sample the latent space parameter distribution to generate a compressed version of the input video.

Abstract: Methods are provided for selecting pairs of images from which to generate simulated video sequences by interpolating between the pairs of images. A variety of filters are provided by assessing the quality of such image pairs, such that the quality of the simulated video generated by pairs of images selected thereby is improved. These filters can be performed sequentially, with subsequent filters only executed if all preceding filters have ‘passed,’ thereby reducing the computational cost. Some of the filters include generating, for a pair of images, respective embeddings of the images into a representational space and then comparing the embeddings. Some of the filters include generated a test interpolation image between the pair of images and then assessing a similarity between the test image and the pair of images. Some of the filters include determining optical flow between the pair of images.

Abstract: Apparatuses, systems, and techniques for texture synthesis from small input textures in images using convolutional neural networks. In at least one embodiment, one or more convolutional layers are used in conjunction with one or more transposed convolution operations to generate a large textured output image from a small input textured image while preserving global features and texture, according to various novel techniques described herein.

Abstract: Provided is a computer system that includes one or more processors and one or more non-transitory computer-readable media that collectively store a machine-learned image interpolation model. The machine-learned image interpolation model is configured to: extract, for each of multiple different scales, a respective set of feature values from each of a pair of input images; generate, for each of the multiple different scales, a respective flow estimate for each of the pair of input images that indicates a respective flow from the interpolation time to the respective capture time; warp, for each of the multiple different scales, the respective set of feature values for each of the pair of input images according to the respective flow estimate to generate respective warped sets of features; and generate a interpolated image based on the respective warped sets of features for the pair of input images and the multiple different scales.

Abstract: Apparatuses, systems, and techniques for texture synthesis from small input textures in images using convolutional neural networks. In at least one embodiment, one or more convolutional layers are used in conjunction with one or more transposed convolution operations to generate a large textured output image from a small input textured image while preserving global features and texture, according to various novel techniques described herein.

Abstract: Apparatuses, systems, and techniques to generate interpolated video frames. In at least one embodiment, an interpolated video frame is generated based, at least in part, on a first set of pixel data sampled from a first video frame, and a second set of pixel data sampled from a second video frame based, at least in part, on a set of forward pointing motion vectors from the first video frame to the second video frame.

Abstract: Apparatuses, systems, and techniques to generate interpolated video frames. In at least one embodiment, an interpolated video frame is generated based, at least in part, on one of a plurality of possible motions of one or more objects from a first video frame to a second video frame.

Abstract: Apparatuses, systems, and techniques for texture synthesis from small input textures in images using convolutional neural networks. In at least one embodiment, one or more convolutional layers are used in conjunction with one or more transposed convolution operations to generate a large textured output image from a small input textured image while preserving global features and texture, according to various novel techniques described herein.

Abstract: Apparatuses, systems, and techniques to enhance video. In at least one embodiment, one or more neural networks are used to create, from a first video, a second video having a higher frame rate, higher resolution, or reduced number of missing or corrupt video frames.