Abstract: A mechanism is described for image frame rendering. An apparatus of embodiments, as described herein, includes one or more processors to receive a plurality of past image frames including a plurality of pixels, receive a predicted optical flow, generate a predicted frame and a confidence map associated with the predicted frame based on the plurality of past image frames and the predicted optical flow, render a first set of the plurality of pixels in the predicted frame based on the confidence map and adding the rendered pixels to the predicted frame to generate a final frame.

Abstract: A mechanism is described for image frame rendering. An apparatus of embodiments, as described herein, includes one or more processors to receive a plurality of past image frames including a plurality of pixels, receive a predicted optical flow, generate a predicted frame and a confidence map associated with the predicted frame based on the plurality of past image frames and the predicted optical flow, render a first set of the plurality of pixels in the predicted frame based on the confidence map and adding the rendered pixels to the predicted frame to generate a final frame.

Abstract: One embodiment provides a graphics processor comprising a base die including a plurality of chiplet sockets and a plurality of chiplets coupled with the plurality of chiplet sockets. At least one of the plurality of chiplets include a graphics core cluster including a plurality of processing elements, a shared local memory coupled with the plurality of processing elements, a plurality of matrix engines coupled with the shared local memory, and codec circuitry coupled with the shared local memory and the plurality of matrix engines. The codec circuitry is configured to decode matrix data stored in the shared local memory in a first format into a second format for consumption by the plurality of matrix engines.

Abstract: Embodiments described herein provide techniques to facilitate hierarchical scaling when quantizing neural network data to a reduced-bit representation. The techniques includes operations to load a hierarchical scaling map for a tensor associated with a neural network, partition the tensor into a plurality of regions that respectively include one or more subregions based on the hierarchical scaling map, hierarchically scale numerical values of the tensor based on a first scale factor and second scale factor via the matrix accelerator circuitry, the first scale factor based on a statistical measure of a subregion of numerical values of within a region of the plurality of regions and the second scale factor based on a statistical measure of the region that includes the subregion, and generate a quantized representation of the tensor via quantization of hierarchically scaled numerical values.

Abstract: Systems and methods are for reduced precision models for generative graphics are provided. In one example, information is received that is indicative of saliency of one or more portions of an image that is to be generated. The image is then generated by, for each portion of the one or more portions of the image: (i) based on the information indicative of saliency, selecting a generative model from among multiple pre-trained generative models (e.g., diffusion models) quantized at different precision levels (e.g., using various weights of between 32-bits and 1-bit, inclusive); and (ii) applying the selected generative model to pixels associated with the portion.

Abstract: One embodiment provides a multi-chiplet graphics processor comprising a plurality of chiplets, where a chiplet of the plurality of chiplets comprise a memory interface, processing resources configured to execute threads of a kernel, and thread dispatch circuitry to facilitate dispatch of threads of the kernel to the processing resources. The processing resources are configured to execute threads of a first kernel, receive dispatch of threads of a second kernel for execution before completion of the first kernel as threads of the first kernel retire, execute a first phase of the second kernel during completion of execution of the first kernel, via a thread of the first kernel, signal an event via an uncached write to a global memory, and execute a second phase of the second kernel based on detection of the event via an uncached read from the global memory.

Abstract: One embodiment provides a graphics processor comprising a block of graphics cores and circuitry including a programmable neural network unit, the programmable neural network unit including one or more neural network hardware blocks, wherein a neural network hardware block includes circuitry to perform neural network operations and activation operations for a layer of a neural network, the programmable neural network unit addressable by cores within the block of graphics cores, wherein the programmable neural network unit is to configure one or more neural network hardware blocks with a meta-shader neural network, the meta-shader neural network to generate a texture for one of multiple types of terrain.

Abstract: One embodiment provides a graphics processor comprising a block of execution resources, a cache memory, a cache memory prefetcher, and circuitry including a programmable neural network unit, the programmable neural network unit comprising a network hardware block including circuitry to perform neural network operations and activation operations for a layer of a neural network, the programmable neural network unit addressable by cores within the block of graphics cores and the neural network hardware block configured to perform operations associated with a neural network configured to determine a prefetch pattern for the cache memory prefetcher.

Abstract: A system and method for foveated stereo rendering.

Abstract: Described herein are techniques to enhance the user experience for 3D rendered applications via neural frame generation using upsampled optical flow data. In one embodiment, a neural network is trained using both sparse optical flow data and dense optical flow data to enable neural frame generation to be performed by a deployed neural network using only sparse optical flow data. The sparse optical flow data can be upsampled to dense optical flow data by the trained neural network. The neural network can then use the upsampled dense optical flow data to perform frame generation.

Abstract: Described herein are techniques to enhance the user experience for 3D rendered applications via neural frame generation and neural supersampling. One embodiment provides a latency aware unified neural network for frame interpolation and extrapolation. This unified neural network merges interpolation and extrapolation networks into one generalized network that can be applied to both interpolation and extrapolation, depending on the acceptable latency of performance. A further embodiment provides hardware-efficient and latency-aware spatiotemporal neural frame prediction. Hardware-efficient and latency-aware spatiotemporal neural frame prediction enables both frame generation and machine learning supersampling using a single network, rather than using separate networks for frame generation and supersampling.

Abstract: One embodiment provides a graphics processor comprising a block of execution resources, a cache memory, a cache memory prefetcher, and circuitry including a programmable neural network unit, the programmable neural network unit comprising a network hardware block including circuitry to perform neural network operations and activation operations for a layer of a neural network, the programmable neural network unit addressable by cores within the block of graphics cores and the neural network hardware block configured to perform operations associated with a neural network configured to determine a prefetch pattern for the cache memory prefetcher.

Abstract: An apparatus and method for efficiently improving virtual/real interactions in augmented reality. For example, one embodiment of a method comprises: capturing a raw image including depth data; identifying one or more regions of interest based on a detected spatial proximity of one or more virtual objects and one or more real objects; generating a super-resolution map of the one or more regions of interest using machine-learning techniques or results thereof; detecting interactions between the virtual objects and the real objects using the super-resolution map; and performing one or more graphics processing or general purpose processing operations based on the detected interactions.

Abstract: One embodiment provides for a graphics processor comprising a block of graphics compute units, a graphics processor pipeline coupled to the block of graphics compute units, and a programmable neural network unit including one or more neural network hardware blocks. The programmable neural network unit is coupled with the block of graphics compute units and the graphics processor pipeline. The one or more neural network hardware blocks include hardware to perform neural network operations and activation operations for a layer of a neural network. The programmable neural network unit can configure settings of one or more hardware blocks within the graphics processor pipeline based on a machine learning model trained to optimize performance of a set of workloads.

Abstract: Methods, systems and apparatuses may provide for technology that detects virtual background content and real-time foreground content associated with a video session, wherein the real-time foreground content depicts a plurality of participants from different physical environments. The technology may also apply a visual correction to one or more of the real-time foreground content or the virtual background content, wherein the visual correction reduces a difference between the real-time foreground content and the virtual background content with respect to one or more lighting parameters and two or more of the plurality of participants. Additionally, the technology may generate a composite result based on the real-time foreground content, the virtual background content, and the visual correction.

Abstract: Embodiments are directed to neural network processing for multi-object three-dimensional (3D) modeling. An embodiment of a computer-readable storage medium includes executable computer program instructions for obtaining data from multiple cameras, the data including multiple images, and generating a 3D model for 3D imaging based at least in part on the data from the cameras, wherein generating the 3D model includes one or more of performing processing with a first neural network to determine temporal direction based at least in part on motion of one or more objects identified in an image of the multiple images or performing processing with a second neural network to determine semantic content information for an image of the multiple images.

Abstract: A mechanism is described for image frame rendering. An apparatus of embodiments, as described herein, includes one or more processors to receive a plurality of past image frames including a plurality of pixels, receive a predicted optical flow, generate a predicted frame and a confidence map associated with the predicted frame based on the plurality of past image frames and the predicted optical flow, render a first set of the plurality of pixels in the predicted frame based on the confidence map and adding the rendered pixels to the predicted frame to generate a final frame.

Abstract: An apparatus and method for efficiently improving virtual/real interactions in augmented reality. For example, one embodiment of a method comprises: capturing a raw image including depth data; identifying one or more regions of interest based on a detected spatial proximity of one or more virtual objects and one or more real objects; generating a super-resolution map of the one or more regions of interest using machine-learning techniques or results thereof; detecting interactions between the virtual objects and the real objects using the super-resolution map; and performing one or more graphics processing or general purpose processing operations based on the detected interactions.

Abstract: One embodiment provides for a graphics processor comprising a block of graphics compute units, a graphics processor pipeline coupled to the block of graphics compute units, and a programmable neural network unit including one or more neural network hardware blocks. The programmable neural network unit is coupled with the block of graphics compute units and the graphics processor pipeline. The one or more neural network hardware blocks include hardware to perform neural network operations and activation operations for a layer of a neural network. The programmable neural network unit can configure settings of one or more hardware blocks within the graphics processor pipeline based on a machine learning model trained to optimize performance of a set of workloads.

Abstract: One embodiment provides for a graphics processor comprising a block of graphics compute units, a graphics processor pipeline coupled to the block of graphics compute units, and a programmable neural network unit including one or more neural network hardware blocks. The programmable neural network unit is coupled with the block of graphics compute units and the graphics processor pipeline. The one or more neural network hardware blocks include hardware to perform neural network operations and activation operations for a layer of a neural network. The programmable neural network unit can configure settings of one or more hardware blocks within the graphics processor pipeline based on a machine learning model trained to optimize performance of a set of workloads.