Abstract: An encoder of a multimedia system encodes data representative of each image in a video stream to form a bitstream that is transmitted over a network to a decoder that decodes the bitstream and provides the decoded information to a multimedia application for display to the user. As consecutive images may have nearly identical pixel values, the multimedia system detects pixel value variations between consecutive images that are below a specified threshold to reduce active processing on such pictures, which includes, for example, encoding, decoding, and post-processing. The multimedia system either selectively encodes or selectively processes the current image that has pixel values that vary from an immediately preceding image within the specified threshold.

Abstract: Systems, apparatuses, and methods for block type prediction leveraging block-based pixel activities are disclosed. A pre-encoder generates predictions of block types for the blocks of a video frame based on associated pixel activities. For each block, the pre-encoder calculates the difference between the pixel activities of the block of a current frame and the pixel activities of a corresponding block of a previous video frame. If the difference is less than a first threshold, the pre-encoder predicts that the block will be a skip block. If the difference is in between the first threshold and a second threshold, the pre-encoder predicts that the block will be a P-block. Otherwise, if the difference is greater than the second threshold, then the pre-encoder predicts that the block will be an I-block. The pre-encoder uses the predictions to select quantization parameter (QP) ranges for encoding the blocks of the video frame.

Abstract: Systems, apparatuses, and methods for generating a model for determining a quantization strength to use when encoding video frames are disclosed. A pre-encoder performs multiple encoding passes using different quantization strengths on a portion or the entirety of one or more pre-processed video frames. The pre-encoder captures the bit-size of the encoded output for each of the multiple encoding passes. Then, based on the multiple encoding passes, the pre-encoder generates a model for mapping bit-size to quantization strength for encoding video frames or portion(s) thereof. When the encoder begins the final encoding pass for one or more given video frames or any portion(s) thereof, the encoder uses the model to map a preferred bit-size to a given quantization strength. The encoder uses the given quantization strength when encoding the given video frame(s) or frame portion(s) to meet a specified bit-rate for the encoded bitstream.

Abstract: A system and method for dynamically changing encode quality at a block level based on runtime pre-encoding analysis of content in a video stream. A video encoder continuously analyzes the content during runtime, and collects statistics and/or characteristics of the content before encoding it. This classifies the block among pre-defined categories of content, where every category has its own compression parameters.

Abstract: Described is a system and method for dynamically changing encode quality at a block level based on runtime pre-encoding analysis of content in a video stream. A video encoder continuously analyzes the content during runtime, and collects statistics and/or characteristics of the content before encoding it. This classifies the block among pre-defined categories of content, where every category has its own compression parameters.

Abstract: Systems, apparatuses, and methods for implementing spatial block-level pixel activity extraction optimization leveraging motion vectors are disclosed. Control logic coupled to an encoder generates block-level pixel activity metrics for a new frame based on the previously calculated block-level pixel activity data from a reference frame. A cost is calculated for each block of a new frame with respect to a corresponding block of the reference frame. If the cost is less than a first threshold, then the control logic generates an estimate of a pixel activity metric for the block which is equal to a previously calculated pixel activity metric for a corresponding block of the reference frame. If the cost is greater than the first threshold but less than a second threshold, an estimate of the pixel activity metric is generated by extrapolating from the previously calculated pixel activity metric.

Abstract: An encoder calculates a first local encoding parameter for a first block of video content based on one or more local metrics. The encoder modifies the first local encoding parameter based on one or more second local encoding parameters for one or more second blocks of video content that are adjacent the first block of video content. The encoder that encodes the first block using the modified first local encoding parameter. In some cases, the local encoding parameters are quantization parameters used to quantize values of pixels or compression parameters used to compress values of the pixels. The local metric can include one or more of a target bit rate, a texture complexity, a contrast, an indicator of motion in the first block, and an importance map.

Abstract: Systems, apparatuses, and methods for rendering images directly to a video encoder are disclosed. A game engine includes an embedded rendering unit configured to render images in different color spaces depending on the mode. The rendering unit renders images for a first color space only to be driven directly to a display while operating in a first mode. The rendering unit renders images for a second color space only which are provided directly to a video encoder while operating in a second mode. In a third mode, the rendering unit renders images for both color spaces. In one embodiment, the first color space is RGB and the second color space is YUV. The game engine also generates a plurality of attributes associated with each rendered image and the video encoder encodes each rendered image into an encoded bitstream based on the attributes associated with the rendered image.

Abstract: A processing system estimates motion between blocks of images based on differences in pixel activities between blocks. Blocks having similar pixel activities are more likely to be matches for motion estimation than are blocks having dissimilar pixel activities. Accordingly, the processing system compares pixel activity between current block of a current image and candidate blocks within a search area in the reference image, and estimates motion based on a subset of candidate blocks having a difference in pixel activity from that of the current block within a threshold.

Abstract: A multimedia system allocates, during encoding of a multimedia stream, bits to portions of frames based on quality metrics and bit usages for different quantization parameters (QPs). An encoder of the multimedia system encodes a frame in a first pass with a first QP and in a second pass with a second QP. A comparator of the multimedia system measures and compares quality metrics, such as mean squared error, for each portion of the frame encoded with the first QP and the second QP. The comparator compares the difference between the quality metrics for each portion encoded with each QP to a threshold. If the difference in quality metrics for a portion exceeds the threshold, the comparator selects the portion for inclusion in a subset of portions to be encoded with the second QP.

Abstract: Systems, apparatuses, and methods for encoding bitstreams of uniquely rendered video frames with variable frame rates are disclosed. A rendering unit and an encoder in a server are coupled via a network to a client with a decoder. The rendering unit dynamically adjusts the frame rate of uniquely rendered frames. Depending on the operating mode, the rendering unit conveys a constant frame rate to the encoder by repeating some frames or the rendering unit conveys a variable frame rate to the encoder by conveying only uniquely rendered frames to the encoder. Depending on the operating mode, the encoder conveys a constant frame rate bitstream to the decoder by encoding repeated frames as skip frames, or the encoder conveys a variable frame rate bitstream to the decoder by dropping repeated frames from the bitstream.

Abstract: A feedback processing module includes a memory configured to store feedback received from an encoder. The feedback includes parameters associated with encoded graphics content generated by a graphics engine. The feedback processing module also includes a processor configured to generate configuration information for the graphics engine based on the feedback. The graphics engine is configured to execute a workload based on the configuration information. In some cases, the feedback processing module is also configured to receive feedback from a decoder that is used to decode the graphics content that is encoded by the encoder and generate the configuration information based on the feedback received from the decoder.

Abstract: The present disclosure is directed to techniques for determining a perceptual importance map. The perceptual importance map indicates the relative importance to the human visual system of different portions of an image. The techniques include obtaining cost values for the blocks of an image, where cost values are values used in determining motion vectors. For each block, a confidence value is derived from the cost values. The confidence value indicates the confidence with which the motion vector is believed to be correct. A perceptual importance value is determined based on the confidence value via one or more modifications to the confidence value to better reflect importance to the human visual system. The generated perceptual importance values can be used for various purposes such as allocating bits for encoding, identifying regions of interest, or selectively rendering portions of an image with greater or lesser detail based on relative perceptual importance.

Abstract: A video server generates metadata representative of interpolation parameters for portions of a first frame representative of a scene in a stream of frames including the first frame. The interpolation parameters are used to generate at least one interpolated frame representative of the scene subsequent to the first frame and prior to a second frame in the stream of frames. The video server incorporates the metadata into the stream and transmits the stream including the multiplexed metadata. A video client receives the first frame representative the stream of frames including the metadata. The video client generates one or more interpolated frames representative of the scene subsequent to the first frame and prior to a second frame in the stream of frames based on the first frame and the metadata. The video client displays the first frame, the one or more interpolated frames, and the second frame.

Abstract: An encoder of a multimedia system encodes data representative of each image in a video stream to form a bitstream that is transmitted over a network to a decoder that decodes the bitstream and provides the decoded information to a multimedia application for display to the user. As consecutive images may have nearly identical pixel values, the multimedia system detects pixel value variations between consecutive images that are below a specified threshold to reduce active processing on such pictures, which includes, for example, encoding, decoding, and post-processing. The multimedia system either selectively encodes or selectively processes the current image that has pixel values that vary from an immediately preceding image within the specified threshold.

Abstract: Systems, apparatuses, and methods for rendering images directly to a video encoder are disclosed. A game engine includes an embedded rendering unit configured to render images in different color spaces depending on the mode. The rendering unit renders images for a first color space only to be driven directly to a display while operating in a first mode. The rendering unit renders images for a second color space only which are provided directly to a video encoder while operating in a second mode. In a third mode, the rendering unit renders images for both color spaces. In one embodiment, the first color space is RGB and the second color space is YUV. The game engine also generates a plurality of attributes associated with each rendered image and the video encoder encodes each rendered image into an encoded bitstream based on the attributes associated with the rendered image.

Abstract: Systems, apparatuses, and methods for encoding bitstreams of uniquely rendered video frames with variable frame rates are disclosed. A rendering unit and an encoder in a server are coupled via a network to a client with a decoder. The rendering unit dynamically adjusts the frame rate of uniquely rendered frames. Depending on the embodiment, the rendering unit conveys a constant frame rate to the encoder by repeating some frames or the rendering unit conveys a variable frame rate to the encoder by conveying only uniquely rendered frames to the encoder. Depending on the embodiment, the encoder conveys a constant frame rate bitstream to the decoder by encoding repeated frames as skip frames, or the encoder conveys a variable frame rate bitstream to the decoder by dropping repeated frames from the bitstream.

Abstract: A processing device for use with a video conferencing network is provided. The processing device includes memory configured to store data and a processor. The processor is configured to determine a first sampling phase for a portion of first video data and chrominance sub-sample the portion of first video data using the first sampling phase. The processor is also configured to encode the sub-sampled portion of first video data and decode a sub-sampled, encoded portion of second video data. The processor is further configured to determine a second sampling phase at which the portion of second video data is chrominance sub-sampled and chrominance up-sample the portion of second video data using the second sample phase.

Abstract: A processing system selectively renders pixels or blocks of pixels of an image and leaves some pixels or blocks of pixels unrendered to conserve resources. The processing system generates a motion vector field to identify regions of an image having moving areas. The processing system uses a rendering processor to identify as regions of interest those units having little to no motion, based on the motion vector field, and a large amount of edge activity, and to minimize the probability of unrendered pixels, or “holes”, in these regions. To avoid noticeable patterns, the rendering processor applies a probability map to determine the possible locations of holes, assigning to each unit a probability indicating the percentage of pixels within the unit that will be holes, and assigning a lower probability to units identified as regions of interest.

Abstract: Systems, apparatuses, and methods for adaptive use-case based filtering of compressed video streams are disclosed. In one embodiment, a system includes at least a display and a processor coupled to at least one memory device. The system is configured to receive a compressed video stream. For each received frame of the compressed video stream, the system decompresses the compressed video frame into an unfiltered frame. Then, the system can utilize a first filter to filter the unfiltered frame to generate a filtered frame. In one embodiment, the first filter is a de-blocking filter (DBF) combined with a sample adaptive offset (SAO) filter. Also, in this embodiment, the first filter is compliant with a video compression standard. The unfiltered frame and the filtered frame are provided as inputs to a second filter which performs a use-case specific de-noising of the inputs to generate a de-noised frame with reduced artifacts.