Abstract: An encoder encodes pixels representative of a picture in a multimedia stream, generates a first approximate signature based on approximate values of pixels in a reconstructed copy of the picture, and transmits the encoded pixels and the first approximate signature. A decoder receives a first packet including the encoded pixels and the first approximate signature, decodes the encoded pixels, and transmits a first signal in response to comparing the first approximate signature and a second approximate signature generated based on approximate values of the decoded pixels. If a corrupted packet is detected, the multimedia application requests an intra-coded picture in response to the first approximate signature differing from the second approximate signature. The second signal instructs the decoder to bypass requesting an intra-coded picture and to continue decoding received packets in response to the first approximate signature being equal to the second approximate signature.

Abstract: A processing apparatus is provided that includes an encoder configured to encode current frames of video data using previously encoded reference frames and perform motion searches within a search window about each of a plurality of co-located portions of a reference frame. The processing apparatus also includes a processor configured to determine, prior to performing the motion searches, which locations of the reference frame to reload the search window according to a threshold number of search window reloads using predicted motions of portions of the reference frame corresponding to each of the locations. The processor is also configured to cause the encoder to reload the search window at the determined locations of the reference frame and, for each of the remaining locations of the reference frame, slide the search window in a first direction indicated by the location of the next co-located portion of the reference frame.

Abstract: Systems, methods and apparatuses of processing data of a VR system are disclosed that comprise receiving tracking information which includes at least one of user position information and eye gaze point information. One or more processors may be used to predict, based on the user tracking information, a user viewpoint of a next frame of a sequence of frames of video data to be displayed. Using the prediction, a portion of the next frame of video data to be displayed is rendered at an estimated location in the next frame. A corresponding matching portion in a previously encoded frame is determined based on the estimated location of the portion in the next frame and the portion of the next frame of video data is encoded.

Abstract: An encoder encodes pixels representative of a picture in a multimedia stream, generates a first approximate signature based on approximate values of pixels in a reconstructed copy of the picture, and transmits the encoded pixels and the first approximate signature. A decoder receives a first packet including the encoded pixels and the first approximate signature, decodes the encoded pixels, and transmits a first signal in response to comparing the first approximate signature and a second approximate signature generated based on approximate values of the decoded pixels. If a corrupted packet is detected, the multimedia application requests an intra-coded picture in response to the first approximate signature differing from the second approximate signature. The second signal instructs the decoder to bypass requesting an intra-coded picture and to continue decoding received packets in response to the first approximate signature being equal to the second approximate signature.

Abstract: Systems, apparatuses, and methods for rendering images directly to a video encoder are disclosed. A game engine includes an embedded rendering unit configured to render images in different color spaces depending on the mode. The rendering unit renders images for a first color space only to be driven directly to a display while operating in a first mode. The rendering unit renders images for a second color space only which are provided directly to a video encoder while operating in a second mode. In a third mode, the rendering unit renders images for both color spaces. In one embodiment, the first color space is RGB and the second color space is YUV. The game engine also generates a plurality of attributes associated with each rendered image and the video encoder encodes each rendered image into an encoded bitstream based on the attributes associated with the rendered image.

Abstract: An encoder of a multimedia system encodes data representative of each image in a video stream to form a bitstream that is transmitted over a network to a decoder that decodes the bitstream and provides the decoded information to a multimedia application for display to the user. As consecutive images may have nearly identical pixel values, the multimedia system detects pixel value variations between consecutive images that are below a specified threshold to reduce active processing on such pictures, which includes, for example, encoding, decoding, and post-processing. The multimedia system either selectively encodes or selectively processes the current image that has pixel values that vary from an immediately preceding image within the specified threshold.

Abstract: Systems, apparatuses, and methods for encoding bitstreams of uniquely rendered video frames with variable frame rates are disclosed. A rendering unit and an encoder in a server are coupled via a network to a client with a decoder. The rendering unit dynamically adjusts the frame rate of uniquely rendered frames. Depending on the embodiment, the rendering unit conveys a constant frame rate to the encoder by repeating some frames or the rendering unit conveys a variable frame rate to the encoder by conveying only uniquely rendered frames to the encoder. Depending on the embodiment, the encoder conveys a constant frame rate bitstream to the decoder by encoding repeated frames as skip frames, or the encoder conveys a variable frame rate bitstream to the decoder by dropping repeated frames from the bitstream.

Abstract: Encoding video data includes accessing video information, and encoding macroblocks in an intra refresh (IR) region of a video frame of the video information with restricted quantization parameters (QPs) based on QPs of macroblocks within at least one of a non-IR region of the video frame and a co-located region of a neighboring video frame in a temporal domain.

Abstract: A processing system selectively renders pixels or blocks of pixels of an image and leaves some pixels or blocks of pixels unrendered to conserve resources. The processing system generates a motion vector field to identify regions of an image having moving areas. The processing system uses a rendering processor to identify as regions of interest those units having little to no motion, based on the motion vector field, and a large amount of edge activity, and to minimize the probability of unrendered pixels, or “holes”, in these regions. To avoid noticeable patterns, the rendering processor applies a probability map to determine the possible locations of holes, assigning to each unit a probability indicating the percentage of pixels within the unit that will be holes, and assigning a lower probability to units identified as regions of interest.

Abstract: Systems, apparatuses, and methods for adaptive use-case based filtering of compressed video streams are disclosed. In one embodiment, a system includes at least a display and a processor coupled to at least one memory device. The system is configured to receive a compressed video stream. For each received frame of the compressed video stream, the system decompresses the compressed video frame into an unfiltered frame. Then, the system can utilize a first filter to filter the unfiltered frame to generate a filtered frame. In one embodiment, the first filter is a de-blocking filter (DBF) combined with a sample adaptive offset (SAO) filter. Also, in this embodiment, the first filter is compliant with a video compression standard. The unfiltered frame and the filtered frame are provided as inputs to a second filter which performs a use-case specific de-noising of the inputs to generate a de-noised frame with reduced artifacts.

Abstract: A processing apparatus is provided that includes an encoder configured to encode current frames of video data using previously encoded reference frames and perform motion searches within a search window about each of a plurality of co-located portions of a reference frame. The processing apparatus also includes a processor configured to determine, prior to performing the motion searches, which locations of the reference frame to reload the search window according to a threshold number of search window reloads using predicted motions of portions of the reference frame corresponding to each of the locations. The processor is also configured to cause the encoder to reload the search window at the determined locations of the reference frame and, for each of the remaining locations of the reference frame, slide the search window in a first direction indicated by the location of the next co-located portion of the reference frame.

Abstract: Described is a system and method for dynamically changing a resolution level at a frame level based on runtime pre-encoding analysis of content in a video stream. A video encoder continuously analyzes the content during runtime, and collects statistics and/or characteristics of the content before encoding it. This classifies the frame among pre-defined categories of content, where every category has its own bitrate/resolution relation. The runtime encoding resolution is dynamically dependent on the target bitrate and the collected statistics and/or characteristics of the content. This achieves a high quality encode for sequences that are composed of scenes with various content complexity levels for different frames in the video streams.

Abstract: A method includes intra-refresh encoding each picture of a first set of pictures such that a position of a refresh region for the picture is spatially shifted relative to the position of the refresh region for a previous picture of the first set responsive to determining global motion associated with the first does not exceed a specified threshold. The method further includes intra-refresh encoding each picture of a second set of pictures such that a position of a refresh region for each picture of the second set is fixed to be immediately adjacent to a picture edge that is in a direction of global motion associated with the second set responsive to determining the global motion associated with the second set exceeds the specified threshold.

Abstract: Systems, methods and apparatuses of processing data of a VR system are disclosed that comprise receiving tracking information which includes at least one of user position information and eye gaze point information. One or more processors may be used to predict, based on the user tracking information, a user viewpoint of a next frame of a sequence of frames of video data to be displayed. Using the prediction, a portion of the next frame of video data to be displayed is rendered at an estimated location in the next frame. A corresponding matching portion in a previously encoded frame is determined based on the estimated location of the portion in the next frame and the portion of the next frame of video data is encoded.

Abstract: Systems, apparatuses and methods of adaptively controlling a cache operating voltage are provided that comprise receiving indications of a plurality of cache usage amounts. Each cache usage amount corresponds to an amount of data to be accessed in a cache by one of a plurality of portions of a data processing application. The plurality of cache usage amounts are determining based on the received indications of the plurality of cache usage amounts. A voltage level applied to the cache is adaptively controlled based on one or more of the plurality of determined cache usage amounts. Memory access to the cache is controlled to be directed to a non-failing portion of the cache at the applied voltage level.

Abstract: Various codecs and methods of using the same are disclosed. In one aspect, a method of processing video data is provided that includes encoding or decoding the video data with a codec in aggressive deployment and correcting one or more errors in the encoding or decoding wherein the error correction includes re-encoding or re-decoding the video data in a non-aggressive deployment or generating a skip picture.

Abstract: An encoder encodes pixels representative of a picture in a multimedia stream, generates a first approximate signature based on approximate values of pixels in a reconstructed copy of the picture, and transmits the encoded pixels and the first approximate signature. A decoder receives a first packet including the encoded pixels and the first approximate signature, decodes the encoded pixels, and transmits a first signal in response to comparing the first approximate signature and a second approximate signature generated based on approximate values of the decoded pixels. If a corrupted packet is detected, the multimedia application requests an intra-coded picture in response to the first approximate signature differing from the second approximate signature. The second signal instructs the decoder to bypass requesting an intra-coded picture and to continue decoding received packets in response to the first approximate signature being equal to the second approximate signature.

Abstract: The present disclosure is directed a system and method for exploiting camera and depth information associated with rendered video frames, such as those rendered by a server operating as part of a cloud gaming service, to more efficiently encode the rendered video frames for transmission over a network. The method and system of the present disclosure can be used in a server operating in a cloud gaming service to improve, for example, the amount of latency, downstream bandwidth, and/or computational processing power associated with playing a video game over its service. The method and system of the present disclosure can be further used in other applications where camera and depth information of a rendered or captured video frame is available.

Abstract: A data structure and method of use thereof for encoding video information are described. Macroblock parameters are initialized, and it is determined whether an operating point is selected. If the operating point is selected, then the following occurs: each quad of nodes of a first node level are obtained and a check for merger is done on them; each quad of nodes of a second node level is obtained and a check for merger is done on them; nodes of a third node level are obtained and check for merger is done on them; nodes of a fourth node level are obtained and a check for merger is done on them; and modes are assigned responsive to cost of combinations of encoding modes associated with possible mergers.