Abstract: A computing system generates a trained super-resolution model using correlated high-resolution and low-resolution image pairs. A first image generator produces high-resolution images, which are processed by a downscaler network to create corresponding low-resolution images. The downscaler network is trained using an initial set of image pairs. Additional high-resolution images are generated and processed by the downscaler network to produce additional low-resolution images, forming a second set of image pairs. The super-resolution model is trained using the second set of image pairs. The trained super-resolution model processes low-resolution images to generate enhanced images with improved detail and reduced artifacts. The model can be used to increase resolution of input images or refine details without changing resolution. The trained model may be deployed on different devices and applied across various rendering environments.

Abstract: Innovations in machine learning (“ML”) networks used in video processing scenarios are described. For example, an ML refinement network can be used to refine video after a video decoder has reconstructed the video. Using the ML refinement network for post-processing can mitigate compression artifacts introduced during encoding and otherwise improve the quality of the reconstructed video. Or, as another example, an ML encoder network and ML decoder network can be used, in combination with a core video encoder and core video decoder, for hybrid compression and corresponding decompression. In the hybrid compression, the ML encoder network can transform video before encoding in order to boost rate-distortion performance of the core video encoder.

Abstract: Phased loss function training of machine learning models that are configured for performing super-resolution image processing includes training with two or more discrete phases that each apply different combinations of loss function training to the super-resolution models. In a first phase, for example, a first loss function is applied to the model that is a per-pixel or a non-perceptual loss function. In a second phase, which begins after the model reaches a threshold of convergence in the first phase, a blended loss function is applied, which includes the application of at least one perceptual loss function. Then, after the model reaches a subsequent threshold of convergence in the second phase, which is greater than the threshold of convergence in the first phase, the model is optionally further modified with optimizations such as quantization and/or sparsity optimizations for further facilitating subsequent super-resolution processing.

Abstract: Innovations in machine learning (“ML”) networks used in video processing scenarios are described. For example, an ML refinement network can be used to refine video after a video decoder has reconstructed the video. Using the ML refinement network for post-processing can mitigate compression artifacts introduced during encoding and otherwise improve the quality of the reconstructed video. Or, as another example, an ML encoder network and ML decoder network can be used, in combination with a core video encoder and core video decoder, for hybrid compression and corresponding decompression. In the hybrid compression, the ML encoder network can transform video before encoding in order to boost rate-distortion performance of the core video encoder.

Abstract: Innovations in machine learning (“ML”) networks used in video processing scenarios are described. For example, an ML refinement network can be used to refine video after a video decoder has reconstructed the video. Using the ML refinement network for post-processing can mitigate compression artifacts introduced during encoding and otherwise improve the quality of the reconstructed video. Or, as another example, an ML encoder network and ML decoder network can be used, in combination with a core video encoder and core video decoder, for hybrid compression and corresponding decompression. In the hybrid compression, the ML encoder network can transform video before encoding in order to boost rate-distortion performance of the core video encoder.

Abstract: Systems are provided for generating training data from images that are obtained from image generators that are typically configured to only generate a single image per frame. The image generators are modified or otherwise controlled to generate two different images at different resolutions for each of a plurality of frames. The training data is created by pairing the low-resolution images and high-resolution images for common frames into training data set pairings. A super-resolution model is applied to the training data set pairings to create a trained super-resolution model.

Abstract: Phased loss function training of machine learning models that are configured for performing super-resolution image processing includes training with two or more discrete phases that each apply different combinations of loss function training to the super-resolution models. In a first phase, for example, a first loss function is applied to the model that is a per-pixel or a non-perceptual loss function. In a second phase, which begins after the model reaches a threshold of convergence in the first phase, a blended loss function is applied, which includes the application of at least one perceptual loss function. Then, after the model reaches a subsequent threshold of convergence in the second phase, which is greater than the threshold of convergence in the first phase, the model is optionally further modified with optimizations such as quantization and/or sparsity optimizations for further facilitating subsequent super-resolution processing.

Abstract: Systems are provided for generating training data from images that are obtained from image generators that are typically configured to only generate a single image per frame. The image generators are modified or otherwise controlled to generate two different images at different resolutions for each of a plurality of frames. The training data is created by pairing the low-resolution images and high-resolution images for common frames into training data set pairings. A super-resolution model is applied to the training data set pairings to create a trained super-resolution model.

Abstract: Described is an adaptive scheduler associated with a desktop window manager that dynamically controls the rate at which graphics frames are composed. Values corresponding to performance when composing a frame are measured, and the frame composition rate is adjusted as necessary based on the values. The measured data is sampled to provide smooth adjustments. The sampled data is evaluated as to whether the current frame rate is too slow, too fast, or acceptable. If too slow, the frame rate may increased relative to the refresh rate, while if too fast, the frame rate is decreased relative to the refresh rate. In one implementation, the frame rate is too fast if a count of missed frames achieves a missed threshold value, or if a count of late frames achieves a late threshold value. The frame rate is too slow if a count of early frames exceeds an early threshold value.