Abstract: Embodiments herein describe a method for generating multiple neural network model approximations of a compute engine of an integrated circuit (IC) including at least one fault, matching a fault map loaded to the IC with one of the multiple neural network model approximations, and loading a matched neural network model approximation to the IC. The compute engine is a multiply-accumulate (MAC) unit incorporated within an artificial intelligence (AI) accelerator. The multiple neural network model approximations are generated when the compute engine transitions into an approximate mode. In the approximate mode, a first set of operations are substituted for a second set of operations, where the first set of operations are higher precision arithmetic operations and the second set of operations are lower precision arithmetic operations.

Abstract: Systems and methods described herein for encoding of video frames based on an amount of text contained within each block of the frame. For each frame, text detection is performed to identify text contained within each block of the frame and each pixel position within the frame. The text detection outputs, for each block and pixel position, an amount of textual content with respect to a preset threshold amount. Based on the output, only the blocks and pixel positions having an amount of text content equal to or greater than the threshold amount are selected as candidates for block matching. For other blocks block matching processes are bypassed.

Abstract: Methods and apparatus encode image frames using intra-frame prediction by predicting pixels for a block of current pixels, based on a detected spatial pattern of pixel intensity differences among a plurality of neighboring reconstructed pixels to the block of current pixels, and encode a block of pixels of the image frame using the predicted block of reconstructed pixels. Inter-frame prediction is provided by determining whether blocks of pixels in temporally neighboring reconstructed frames corresponding to a candidate motion vector have a pattern of pixel intensity differences among the blocks from temporally neighboring frames. Predicted blocks are produced for a reconstructed frame based on the determined pattern of pixel intensity difference among temporally neighboring frames.

Abstract: Adaptive digital content preprocessing techniques based on a bitrate are described. In an implementation, a parameter of a preprocessing module is set based on a target bitrate. The parameter specifies an amount of preprocessing to be performed in preprocessing digital content. Preprocessed digital content is generated by preprocessing the digital content by the specified amount using the preprocessing module. Encoded digital content is generated by compressing the preprocessed digital content using a compression technique by an encoder. The encoded digital content is then transmitted for communication at the target bitrate.

Abstract: A processing device includes an automated overclocking system and a processor. The automated overclocking system is data-driven and includes an inference engine that executes a machine learning model configured to generate a first output based on a current configuration of the processing device. The first output includes a first set of overclocking parameters. The processor is configured to adjust one or more operating characteristics of at least one component of the processing device based on the first set of overclocking parameters.

Abstract: To leverage an amount of unused bandwidth at a hardware encoder to generate motion estimation data, a processing unit includes a hardware encoder configured to perform a first encoding job including encoder sessions to encode a captured frame, determine motion estimation data for a rendered frame, and encode the rendered frame. Further, the processing unit includes a pre-processing circuitry configured to determine a set of motion estimation parameters based on an encoder delay associated with the performance of the first encoding job by the hardware encoder. The hardware encoder is then configured to perform a second encoding job using the determined set of motion estimation parameters.

Abstract: A processing device includes an automated overclocking system and a processor. The automated overclocking system is data-driven and includes an inference engine that executes a machine learning model configured to generate a first output based on a current configuration of the processing device. The first output includes a first set of overclocking parameters. The processor is configured to adjust one or more operating characteristics of at least one component of the processing device based on the first set of overclocking parameters.

Abstract: To leverage an amount of unused bandwidth at a hardware encoder to generate motion estimation data, a processing unit includes a hardware encoder configured to perform a first encoding job including encoder sessions to encode a captured frame, determine motion estimation data for a rendered frame, and encode the rendered frame. Further, the processing unit includes a pre-processing circuitry configured to determine a set of motion estimation parameters based on an encoder delay associated with the performance of the first encoding job by the hardware encoder. The hardware encoder is then configured to perform a second encoding job using the determined set of motion estimation parameters.

Abstract: An encoder encodes an image portion by recursively partitioning the portion into a partitioning hierarchy of levels. The top level has a single block representing the entire portion and each lower level has four smaller blocks representing a corresponding larger block at a higher level. A palette table is generated for each bottom-level block based on the pixels of the associated block. For each successively higher level, the encoder generates a palette table for each current-level block by selecting palette colors based on the palette colors from the four palette tables for the associated four blocks at the next-lowest level. A color index map is then generated based on a final palette table selected from the palette tables generated for the partitioning hierarchy. A representation of the portion is then encoded using the final palette table and the color index map to generate a corresponding segment of an encoded bitstream.

Abstract: A processing device includes an automated overclocking system and a processor. The automated overclocking system is data-driven and includes an inference engine that executes a machine learning model configured to generate a first output based on a current configuration of the processing device. The first output includes a first set of overclocking parameters. The processor is configured to adjust one or more operating characteristics of at least one component of the processing device based on the first set of overclocking parameters.

Abstract: Systems, apparatuses, and methods for performing parallel histogram calculation with application to palette table derivation are disclosed. An encoder calculates a first histogram for a first portion of pixel component value bits of a block of pixels. Then, the encoder selects a first number of the highest pixel count bins from the first histogram. Also, the encoder calculates a second histogram for a second portion of pixel component value bits of the block. The encoder selects a second number of the highest pixel count bins from the second histogram. A third histogram is calculated from the concatenation of bits assigned to the first and second number of bins, and the highest pixel count bins are selected from the third histogram. A palette table is derived based on these highest pixel count bins selected from the third histogram, and the block of pixels is encoded using the palette table.

Abstract: Systems, apparatuses, and methods for calculating multi-pass histograms for palette table derivation are disclosed. An encoder calculates a first histogram for a first portion of most significant bits (MSBs) of pixel component values of a block of an image or video frame. Then, the encoder selects a given number of the highest pixel count bins from the first histogram. The encoder then increases the granularity of these selected highest pixel count bins by evaluating one or more additional bits from the pixel component values. A second histogram is calculated for the concatenation of the original first portion MSBs from the highest pixel count bins and the one or more additional bits, and the highest pixel count bins are selected from the second histogram. A palette table is derived based on these highest pixel count bins selected from the second histogram, and the block is encoded using the palette table.

Abstract: Methods and apparatus encode image frames using intra-frame prediction by predicting pixels for a block of current pixels, based on a detected spatial pattern of pixel intensity differences among a plurality of neighboring reconstructed pixels to the block of current pixels, and encode a block of pixels of the image frame using the predicted block of reconstructed pixels. Inter-frame prediction is provided by determining whether blocks of pixels in temporally neighboring reconstructed frames corresponding to a candidate motion vector have a pattern of pixel intensity differences among the blocks from temporally neighboring frames. Predicted blocks are produced for a reconstructed frame based on the determined pattern of pixel intensity difference among temporally neighboring frames.

Abstract: Adaptive digital content preprocessing techniques based on a bitrate are described. In an implementation, a parameter of a preprocessing module is set based on a target bitrate. The parameter specifies an amount of preprocessing to be performed in preprocessing digital content. Preprocessed digital content is generated by preprocessing the digital content by the specified amount using the preprocessing module. Encoded digital content is generated by compressing the preprocessed digital content using a compression technique by an encoder. The encoded digital content is then transmitted for communication at the target bitrate.

Abstract: Adaptive decoder-drive encoder reconfiguration techniques are described. In one example, techniques include detecting an operational condition at a consumer using a sensor, the consumer receiving a communication of digital content from an encoder; generating an adaptation instruction by the decoder based on the detecting; transmitting the adaptation instruction by the decoder for receipt by the encoder; and receiving an adapted communication of the digital content generated by the encoder, the adapted communication caused by reconfiguration of the encoder based on the adaptation instruction received from the decoder.

Abstract: A feedback processing module includes a memory configured to store feedback received from an encoder. The feedback includes parameters associated with encoded graphics content generated by a graphics engine. The feedback processing module also includes a processor configured to generate configuration information for the graphics engine based on the feedback. The graphics engine is configured to execute a workload based on the configuration information. In some cases, the feedback processing module is also configured to receive feedback from a decoder that is used to decode the graphics content that is encoded by the encoder and generate the configuration information based on the feedback received from the decoder.

Abstract: Methods and apparatus encode image frames using intra-frame prediction by predicting pixels for a block of current pixels, based on a detected spatial pattern of pixel intensity differences among a plurality of neighboring reconstructed pixels to the block of current pixels, and encode a block of pixels of the image frame using the predicted block of reconstructed pixels. Inter-frame prediction is provided by determining whether blocks of pixels in temporally neighboring reconstructed frames corresponding to a candidate motion vector have a pattern of pixel intensity differences among the blocks from temporally neighboring frames. Predicted blocks are produced for a reconstructed frame based on the determined pattern of pixel intensity difference among temporally neighboring frames.

Abstract: Reference frame detection using sensor metadata, including: storing a plurality of first frames each corresponding to first metadata, wherein the first metadata for each first frame of the plurality of first frames is based on first sensor data from one or more sensors; generating a second frame corresponding to second metadata based on the one or more sensors; identifying, based on the first metadata of the plurality of first frames and the second metadata, a reference frame of the plurality of first frames; and encoding the second frame based on the reference frame.

Abstract: A server offloads graphics effects processing to a client device with graphics processing resources by determining a modification to a graphics effects operation, generating a portion of a rendered video stream using the modification to the graphics effects operation, and providing an encoded representation of the portion of the rendered video stream to the client device, along with metadata representing the modification implemented. The client device decodes the encoded representation to recover the portion of the rendered video stream and selectively performs a graphics effects operation on the recovered portion to at least partially revert the resulting graphics effects for the portion to the intended effects without the modification implemented by the server.

Abstract: Calculating, for each frame of a plurality of frames, a corresponding quality value; calculating, for each frame of the plurality of frames, based on one or more visual attributes of a frame, a weight for the corresponding quality value of the frame; calculating an aggregate quality value for the plurality of frames based on the weight and the corresponding quality value for each frame of the plurality of frames; and providing an assessment of the plurality of frames based on the aggregate quality value for the plurality of frames.