Abstract: Various codecs and methods of using the same are disclosed. In one aspect, a method of processing video data is provided that includes encoding or decoding the video data with a codec in aggressive deployment and correcting one or more errors in the encoding or decoding wherein the error correction includes re-encoding or re-decoding the video data in a non-aggressive deployment or generating a skip picture.

Abstract: An encoder encodes pixels representative of a picture in a multimedia stream, generates a first approximate signature based on approximate values of pixels in a reconstructed copy of the picture, and transmits the encoded pixels and the first approximate signature. A decoder receives a first packet including the encoded pixels and the first approximate signature, decodes the encoded pixels, and transmits a first signal in response to comparing the first approximate signature and a second approximate signature generated based on approximate values of the decoded pixels. If a corrupted packet is detected, the multimedia application requests an intra-coded picture in response to the first approximate signature differing from the second approximate signature. The second signal instructs the decoder to bypass requesting an intra-coded picture and to continue decoding received packets in response to the first approximate signature being equal to the second approximate signature.

Abstract: A processing system filters blocks of a picture to minimize a size and error of the blocks prior to encoding. A pre-processing module of the processing system measures characteristics of a plurality of blocks and evaluates the effects of applying each of a plurality of filters to the blocks prior to encoding in order to predict an increase in compressibility of blocks having similar characteristics that are filtered with each filter before being encoded, with the least impact on quality. The pre-processing module trains models to predict a size and error of blocks filtered with each filter based on block characteristics. The pre-processing module uses the models to calculate a cost in terms of size and error of applying each filter to a given block having certain characteristics. The pre-processing module then applies to the block the filter that is predicted to result in the best cost.

Abstract: An encoder of a multimedia system encodes data representative of each image in a video stream to form a bitstream that is transmitted over a network to a decoder that decodes the bitstream and provides the decoded information to a multimedia application for display to the user. As consecutive images may have nearly identical pixel values, the multimedia system detects pixel value variations between consecutive images that are below a specified threshold to reduce active processing on such pictures, which includes, for example, encoding, decoding, and post-processing. The multimedia system either selectively encodes or selectively processes the current image that has pixel values that vary from an immediately preceding image within the specified threshold.

Abstract: A method includes intra-refresh encoding each picture of a first set of pictures such that a position of a refresh region for the picture is spatially shifted relative to the position of the refresh region for a previous picture of the first set responsive to determining global motion associated with the first does not exceed a specified threshold. The method further includes intra-refresh encoding each picture of a second set of pictures such that a position of a refresh region for each picture of the second set is fixed to be immediately adjacent to a picture edge that is in a direction of global motion associated with the second set responsive to determining the global motion associated with the second set exceeds the specified threshold.

Abstract: Systems, apparatuses, and methods for generating a model for determining a quantization strength to use when encoding video frames are disclosed. A pre-encoder performs multiple encoding passes using different quantization strengths on a portion or the entirety of one or more pre-processed video frames. The pre-encoder captures the bit-size of the encoded output for each of the multiple encoding passes. Then, based on the multiple encoding passes, the pre-encoder generates a model for mapping bit-size to quantization strength for encoding video frames or portion(s) thereof. When the encoder begins the final encoding pass for one or more given video frames or any portion(s) thereof, the encoder uses the model to map a preferred bit-size to a given quantization strength. The encoder uses the given quantization strength when encoding the given video frame(s) or frame portion(s) to meet a specified bit-rate for the encoded bitstream.

Abstract: Systems, apparatuses, and methods for rendering images directly to a video encoder are disclosed. A game engine includes an embedded rendering unit configured to render images in different color spaces depending on the mode. The rendering unit renders images for a first color space only to be driven directly to a display while operating in a first mode. The rendering unit renders images for a second color space only which are provided directly to a video encoder while operating in a second mode. In a third mode, the rendering unit renders images for both color spaces. In one embodiment, the first color space is RGB and the second color space is YUV. The game engine also generates a plurality of attributes associated with each rendered image and the video encoder encodes each rendered image into an encoded bitstream based on the attributes associated with the rendered image.

Abstract: The present disclosure is directed a system and method for exploiting camera and depth information associated with rendered video frames, such as those rendered by a server operating as part of a cloud gaming service, to more efficiently encode the rendered video frames for transmission over a network. The method and system of the present disclosure can be used in a server operating in a cloud gaming service to improve, for example, the amount of latency, downstream bandwidth, and/or computational processing power associated with playing a video game over its service. The method and system of the present disclosure can be further used in other applications where camera and depth information of a rendered or captured video frame is available.

Abstract: Systems, apparatuses, and methods for encoding bitstreams of uniquely rendered video frames with variable frame rates are disclosed. A rendering unit and an encoder in a server are coupled via a network to a client with a decoder. The rendering unit dynamically adjusts the frame rate of uniquely rendered frames. Depending on the operating mode, the rendering unit conveys a constant frame rate to the encoder by repeating some frames or the rendering unit conveys a variable frame rate to the encoder by conveying only uniquely rendered frames to the encoder. Depending on the operating mode, the encoder conveys a constant frame rate bitstream to the decoder by encoding repeated frames as skip frames, or the encoder conveys a variable frame rate bitstream to the decoder by dropping repeated frames from the bitstream.

Abstract: A feedback processing module includes a memory configured to store feedback received from an encoder. The feedback includes parameters associated with encoded graphics content generated by a graphics engine. The feedback processing module also includes a processor configured to generate configuration information for the graphics engine based on the feedback. The graphics engine is configured to execute a workload based on the configuration information. In some cases, the feedback processing module is also configured to receive feedback from a decoder that is used to decode the graphics content that is encoded by the encoder and generate the configuration information based on the feedback received from the decoder.

Abstract: The present disclosure is directed a system and method for exploiting camera and depth information associated with rendered video frames, such as those rendered by a server operating as part of a cloud gaming service, to more efficiently encode the rendered video frames for transmission over a network. The method and system of the present disclosure can be used in a server operating in a cloud gaming service to improve, for example, the amount of latency, downstream bandwidth, and/or computational processing power associated with playing a video game over its service. The method and system of the present disclosure can be further used in other applications where camera and depth information of a rendered or captured video frame is available.

Abstract: Systems, apparatuses, and methods for reducing latency when consuming an encoded video bitstream in real-time are disclosed. A video encoder encodes a video bitstream and writes chunks of the encoded bitstream to a bitstream buffer. Prior to the encoder completing the encoding of an entire frame, or an entire slice of a frame, a consumer module consumes encoded chunks of the bitstream. In one implementation, to enable pipelining of the consumption with the encoding, the encoder updates a buffer write pointer with an indication of the amount of data that has been written to the bitstream buffer. The consumer module retrieves encoded data from the bitstream buffer up to the location indicated by the buffer write pointer. In this way, the consumer module is able to access and consume encoded video data prior to the encoder finishing encoding an entire frame or an entire slice of the frame.

Abstract: Systems, apparatuses and methods of adaptively controlling a cache operating voltage are provided that comprise receiving indications of a plurality of cache usage amounts. Each cache usage amount corresponds to an amount of data to be accessed in a cache by one of a plurality of portions of a data processing application. The plurality of cache usage amounts are determining based on the received indications of the plurality of cache usage amounts. A voltage level applied to the cache is adaptively controlled based on one or more of the plurality of determined cache usage amounts. Memory access to the cache is controlled to be directed to a non-failing portion of the cache at the applied voltage level.

Abstract: An encoder encodes pixels representative of a picture in a multimedia stream, generates a first approximate signature based on approximate values of pixels in a reconstructed copy of the picture, and transmits the encoded pixels and the first approximate signature. A decoder receives a first packet including the encoded pixels and the first approximate signature, decodes the encoded pixels, and transmits a first signal in response to comparing the first approximate signature and a second approximate signature generated based on approximate values of the decoded pixels. If a corrupted packet is detected, the multimedia application requests an intra-coded picture in response to the first approximate signature differing from the second approximate signature. The second signal instructs the decoder to bypass requesting an intra-coded picture and to continue decoding received packets in response to the first approximate signature being equal to the second approximate signature.

Abstract: A processing apparatus is provided that includes an encoder configured to encode current frames of video data using previously encoded reference frames and perform motion searches within a search window about each of a plurality of co-located portions of a reference frame. The processing apparatus also includes a processor configured to determine, prior to performing the motion searches, which locations of the reference frame to reload the search window according to a threshold number of search window reloads using predicted motions of portions of the reference frame corresponding to each of the locations. The processor is also configured to cause the encoder to reload the search window at the determined locations of the reference frame and, for each of the remaining locations of the reference frame, slide the search window in a first direction indicated by the location of the next co-located portion of the reference frame.

Abstract: Systems, methods and apparatuses of processing data of a VR system are disclosed that comprise receiving tracking information which includes at least one of user position information and eye gaze point information. One or more processors may be used to predict, based on the user tracking information, a user viewpoint of a next frame of a sequence of frames of video data to be displayed. Using the prediction, a portion of the next frame of video data to be displayed is rendered at an estimated location in the next frame. A corresponding matching portion in a previously encoded frame is determined based on the estimated location of the portion in the next frame and the portion of the next frame of video data is encoded.

Abstract: An encoder encodes pixels representative of a picture in a multimedia stream, generates a first approximate signature based on approximate values of pixels in a reconstructed copy of the picture, and transmits the encoded pixels and the first approximate signature. A decoder receives a first packet including the encoded pixels and the first approximate signature, decodes the encoded pixels, and transmits a first signal in response to comparing the first approximate signature and a second approximate signature generated based on approximate values of the decoded pixels. If a corrupted packet is detected, the multimedia application requests an intra-coded picture in response to the first approximate signature differing from the second approximate signature. The second signal instructs the decoder to bypass requesting an intra-coded picture and to continue decoding received packets in response to the first approximate signature being equal to the second approximate signature.

Abstract: Systems, apparatuses, and methods for rendering images directly to a video encoder are disclosed. A game engine includes an embedded rendering unit configured to render images in different color spaces depending on the mode. The rendering unit renders images for a first color space only to be driven directly to a display while operating in a first mode. The rendering unit renders images for a second color space only which are provided directly to a video encoder while operating in a second mode. In a third mode, the rendering unit renders images for both color spaces. In one embodiment, the first color space is RGB and the second color space is YUV. The game engine also generates a plurality of attributes associated with each rendered image and the video encoder encodes each rendered image into an encoded bitstream based on the attributes associated with the rendered image.

Abstract: An encoder of a multimedia system encodes data representative of each image in a video stream to form a bitstream that is transmitted over a network to a decoder that decodes the bitstream and provides the decoded information to a multimedia application for display to the user. As consecutive images may have nearly identical pixel values, the multimedia system detects pixel value variations between consecutive images that are below a specified threshold to reduce active processing on such pictures, which includes, for example, encoding, decoding, and post-processing. The multimedia system either selectively encodes or selectively processes the current image that has pixel values that vary from an immediately preceding image within the specified threshold.

Abstract: Systems, apparatuses, and methods for encoding bitstreams of uniquely rendered video frames with variable frame rates are disclosed. A rendering unit and an encoder in a server are coupled via a network to a client with a decoder. The rendering unit dynamically adjusts the frame rate of uniquely rendered frames. Depending on the embodiment, the rendering unit conveys a constant frame rate to the encoder by repeating some frames or the rendering unit conveys a variable frame rate to the encoder by conveying only uniquely rendered frames to the encoder. Depending on the embodiment, the encoder conveys a constant frame rate bitstream to the decoder by encoding repeated frames as skip frames, or the encoder conveys a variable frame rate bitstream to the decoder by dropping repeated frames from the bitstream.