Abstract: Innovations in machine learning (“ML”) networks used in video processing scenarios are described. For example, an ML refinement network can be used to refine video after a video decoder has reconstructed the video. Using the ML refinement network for post-processing can mitigate compression artifacts introduced during encoding and otherwise improve the quality of the reconstructed video. Or, as another example, an ML encoder network and ML decoder network can be used, in combination with a core video encoder and core video decoder, for hybrid compression and corresponding decompression. In the hybrid compression, the ML encoder network can transform video before encoding in order to boost rate-distortion performance of the core video encoder.

Abstract: Innovations in machine learning (“ML”) networks used in video processing scenarios are described. For example, an ML refinement network can be used to refine video after a video decoder has reconstructed the video. Using the ML refinement network for post-processing can mitigate compression artifacts introduced during encoding and otherwise improve the quality of the reconstructed video. Or, as another example, an ML encoder network and ML decoder network can be used, in combination with a core video encoder and core video decoder, for hybrid compression and corresponding decompression. In the hybrid compression, the ML encoder network can transform video before encoding in order to boost rate-distortion performance of the core video encoder.

Abstract: Systems are provided for generating training data from images that are obtained from image generators that are typically configured to only generate a single image per frame. The image generators are modified or otherwise controlled to generate two different images at different resolutions for each of a plurality of frames. The training data is created by pairing the low-resolution images and high-resolution images for common frames into training data set pairings. A super-resolution model is applied to the training data set pairings to create a trained super-resolution model.

Abstract: Phased loss function training of machine learning models that are configured for performing super-resolution image processing includes training with two or more discrete phases that each apply different combinations of loss function training to the super-resolution models. In a first phase, for example, a first loss function is applied to the model that is a per-pixel or a non-perceptual loss function. In a second phase, which begins after the model reaches a threshold of convergence in the first phase, a blended loss function is applied, which includes the application of at least one perceptual loss function. Then, after the model reaches a subsequent threshold of convergence in the second phase, which is greater than the threshold of convergence in the first phase, the model is optionally further modified with optimizations such as quantization and/or sparsity optimizations for further facilitating subsequent super-resolution processing.

Abstract: Systems are provided for generating training data from images that are obtained from image generators that are typically configured to only generate a single image per frame. The image generators are modified or otherwise controlled to generate two different images at different resolutions for each of a plurality of frames. The training data is created by pairing the low-resolution images and high-resolution images for common frames into training data set pairings. A super-resolution model is applied to the training data set pairings to create a trained super-resolution model.

Abstract: Described is an adaptive scheduler associated with a desktop window manager that dynamically controls the rate at which graphics frames are composed. Values corresponding to performance when composing a frame are measured, and the frame composition rate is adjusted as necessary based on the values. The measured data is sampled to provide smooth adjustments. The sampled data is evaluated as to whether the current frame rate is too slow, too fast, or acceptable. If too slow, the frame rate may increased relative to the refresh rate, while if too fast, the frame rate is decreased relative to the refresh rate. In one implementation, the frame rate is too fast if a count of missed frames achieves a missed threshold value, or if a count of late frames achieves a late threshold value. The frame rate is too slow if a count of early frames exceeds an early threshold value.