Abstract: In a computer with a graphics processing unit as a coprocessor of a central processing unit, the graphics processing unit is programmed to perform waves of parallel operations to decode intra-prediction blocks of an image encoded in a certain video coding format. To decode the intra-prediction blocks of an image using the graphics processing unit, the intra-predicted blocks and their reference blocks are identified. The computer identifies whether pixel data from the reference blocks for these intra-predicted blocks are available. Blocks for which pixel data from reference blocks are available are processed in waves of parallel operations on the graphics processing unit as the pixel data becomes available. The process repeats until all intra-predicted blocks are processed. The identification of blocks to process in each wave can be determined by the graphics processing unit or the central processing unit.

Abstract: A video encoding system balances memory usage to store interpolated image data with processing resource usage to interpolate image data without encoding quality degradation or with better encoding quality. This balance can be achieved by identifying and interpolating subregions of a reference image. Each subregion is less than the whole reference image, but larger than a search region for any single block of an image for which motion vectors are to be computed. Each interpolated subregion of the reference image is used to compute motion vectors for multiple blocks of an image being encoded. A video encoding system can identify portions of an image being encoded for which sub-pixel resolution motion vectors are not computed. Motion vectors for such portions of the image can be computed using a reference image without interpolation.

Abstract: A processor that processes encoded media is configured so as to apply constraints to the encoded bitstream. Such constraints are not those required by a specification of a standard with which the encoded media is compliant; instead such constraints reflect portions of the standard that are insufficiently constrained and are applied by the processor to ensure that the processor does not experience performance degradation or errors. The constraints can be applied, for example, as a preprocessing step before reading, writing or decoding the bitstream, or while the bitstream is being decoded, or while the bitstream is being received from a transmission.

Abstract: Innovations in how a host application and video encoder share information and use shared information during video encoding are described. The innovations can help the video encoder perform certain encoding operations and/or help the host application control overall encoding quality and performance. For example, the host application provides regional motion information to the video encoder, which the video encoder can use to speed up motion estimation operations for units of a current picture and more generally improve the accuracy and quality of motion estimation. Or, as another example, the video encoder provides information about the results of encoding the current picture to the host application, which the host application can use to determine when to start a new group of pictures at a scene change boundary. By sharing information in this way, the host application and the video encoder can improve encoding performance, especially for real-time communication scenarios.

Abstract: Techniques are described for split processing of streaming segments in which processing operations are split between a source component and a decoder component. For example, the source component can perform operations for receiving a streaming segment, demultiplexing the streaming segment to separate a video content bit stream, scanning the video content bit stream to find a location at which decoding can begin (e.g., scanning up to a first decodable I-picture, for which header parameter sets are available for decoding), and send the video content bit stream to the decoder component beginning at the location (e.g., the first decodable I-picture). The decoder component can begin decoding at the identified location (e.g., the first decodable I-picture). The decoder component can also discard subsequent pictures that reference a reference picture not present in the video content bit stream (e.g., when decoding starts with a new streaming segment).

Abstract: Techniques are described for remuxing multimedia content received in a digital video broadcasting format without performing transcoding of the video and/or audio content. For example, a computing device with a digital television tuner can receive multimedia content in a digital video broadcast format. The computing device can remux the received multimedia content from the digital video broadcasting format in which the multimedia content is received into a target streaming protocol for streaming to other devices. Remuxing operations can comprise demultiplexing the received multimedia content to separate the audio and video content, performing meta-data reconstruction, and multiplexing the audio and video content into a target stream using a target streaming protocol format.

Abstract: Disclosed herein are innovations in decoding compressed video media data. The disclosed innovations facilitate decoding operations with improved computational efficiency, faster speeds, reduced power, reduced memory usage, and/or reduced latency. In one embodiment, for example, an encoded bitstream of video media data is input from an external video content provider, the encoded bitstream being encoded according to a video codec standard. A decoder is then configured to decode the encoded bitstream based at least in part on supplemental information that identifies a property of the encoded bitstream but that is supplemental to the encoded bitstream (e.g., supplemental information that is not part of the encoded bitstream or its associated media container and that is specific (or related) to the application for which the bitstream is used and/or the standard by which the bitstream is encoded and/or encrypted).

Abstract: Technologies are described herein for providing enhanced packaging, coding, decoding and unpackaging of geometric data. In some configurations, geometric data is obtained by a device. The geometric data is partitioned into data partitions representing reconstruction information for video frames. The data partitions representing frames are then converted and integrated into a network abstraction layer of a bit stream. Geometric data may be obtained from the bit stream by accessing the data partitions from the network abstraction layer. The data partitions can be then processed into geometric data for further processing, such as the reconstruction, generation, display or processing of a three dimensional (3D) object modeled by the geometric data.

Abstract: Disclosed herein are innovations in decoding compressed video media data. The disclosed innovations facilitate decoding operations with improved computational efficiency, faster speeds, reduced power, reduced memory usage, and/or reduced latency. In one embodiment, for example, an encoded bitstream of video media data is input from an external video content provider, the encoded bitstream being encoded according to a video codec standard. A decoder is then configured to decode the encoded bitstream based at least in part on supplemental information that identifies a property of the encoded bitstream but that is supplemental to the encoded bitstream (e.g., supplemental information that is not part of the encoded bitstream or its associated media container and that is specific (or related) to the application for which the bitstream is used and/or the standard by which the bitstream is encoded and/or encrypted).

Abstract: Innovations in the area of hardware-protected digital rights management (“DRM”) systems are presented. For example, a hardware-protected DRM system includes a trusted layer and untrusted layer. In the untrusted layer, a control module receives source media data that includes encrypted media data. The control module processes metadata about the media data. The metadata, possibly exposed by a module in the trusted layer, is not opaque within the untrusted layer. In the trusted layer, using key data, a module decrypts encrypted media data, which can be the encrypted media data from the source media data or a transcripted version thereof. A module in the trusted layer decodes the decrypted media data. A host decoder in the untrusted layer uses the metadata to manage at least some aspects of the decoding, rendering and display in the trusted layer, without exposure of decrypted media data or key data within the untrusted layer.

Abstract: A computing system is disclosed and may include a processor, a plurality of video processing tools, and memory. The plurality of video processing tools include plural alternative video decoder implementations for a first video format, plural alternative video conversion processor implementations, and plural alternative video encoder implementations for a second video format, wherein the plurality of video processing tools form a plurality of transcoding paths. The memory stores instructions for decoding video data in the first video format based on configuration information for the plurality of video processing tools, modifying the decoded video data based on the configuration information for the plurality of video processing tools, and encoding the modified decoded video data into the second video format based on the configuration information for the plurality of video processing tools.

Abstract: A video decoder is disclosed that uses metadata in order to make optimization decisions. In one embodiment, metadata is used to choose which of multiple available decoder engines should receive a video sequence. In another embodiment, the optimization decisions can be based on length and location metadata information associated with a video sequence. Using such metadata information, a decoder engine can skip start-code scanning to make the decoding process more efficient. Also based on the choice of decoder engine, it can decide whether emulation prevention byte removal shall happen together with start code scanning or not.

Abstract: Innovations in encoding and decoding of video pictures in a high-resolution chroma sampling format (such as YUV 4:4:4) using a video encoder and decoder operating on coded pictures in a low-resolution chroma sampling format (such as YUV 4:2:0) are presented. For example, high chroma resolution details are selectively encoded on a region-by-region basis. Or, as another example, coded pictures that contain sample values for low chroma resolution versions of input pictures and coded pictures that contain sample values for high chroma resolution details of the input pictures are encoded as separate sub-sequences of a single sequence of coded pictures, which can facilitate effective motion compensation. In this way, available encoders and decoders operating on coded pictures in the low-resolution chroma sampling format can be effectively used to provide high chroma resolution details.

Abstract: Video content is protected using a digital rights management (DRM) mechanism, the video content having been previously encrypted and compressed for distribution, and also including metadata such as closed captioning data, which might be encrypted or clear. The video content is obtained by a system of a computing device, the metadata is extracted from the video content and provided to a video decoder, and the video content is provided to a secure DRM component. The secure DRM component decrypts the video content and provides the decrypted video content to a secure decoder component of a video decoder. As part of the decryption, the secure DRM component drops the metadata that was included in the obtained video content. However, the video decoder receives the extracted metadata in a non-protected environment, and thus is able to provide the extracted metadata and the decoded video content to a content playback application.

Abstract: A processor that processes encoded media is configured so as to apply constraints to the encoded bitstream. Such constraints are not those required by a specification of a standard with which the encoded media is compliant; instead such constraints reflect portions of the standard that are insufficiently constrained and are applied by the processor to ensure that the processor does not experience performance degradation or errors. The constraints can be applied, for example, as a preprocessing step before reading, writing or decoding the bitstream, or while the bitstream is being decoded, or while the bitstream is being received from a transmission.

Abstract: In one example, a quality management controller of a video processing system may optimize a video recovery action through the selective dropping of video frames. The video processing system may store a compressed video data set in memory. The video processing system may receive a recovery quality indication describing a recovery priority of a user. The video processing system may apply a quality management controller in a video pipeline to execute a video recovery action to retrieve an output data set from the compressed video data set using a video decoder. The quality management controller may select a recovery initiation frame from the compressed video data set to be an initial frame to decompress based upon the recovery quality indication.

Abstract: Buffer optimization techniques are described herein in which a graphics processing system is configured to implement and select between a plurality of buffer schemes for processing of an encoded data stream in dependence upon formats used for decoding and rendering (e.g., video format, bit depth, resolution, content type, etc.) and device capabilities such as available memory and/or processing power. Processing of an encoded data stream for display and rendering via the graphics processing system then occurs using a selected one of the buffer schemes to define buffers employed for the decoding and rendering, including at least configuring the sizes of buffers. The plurality of schemes may include at least one buffer scheme for processing the encoded content when the input format and the output format are the same, and a different buffer scheme for processing the encoded content when the input format and the output format are different.

Abstract: Syntax structures that indicate the completion of coded regions of pictures are described. For example, a syntax structure in an elementary bitstream indicates the completion of a coded region of a picture. The syntax structure can be a type of network abstraction layer unit, a type of supplemental enhancement information message or another syntax structure. For example, a media processing tool such as an encoder can detect completion of a coded region of a picture, then output, in a predefined order in an elementary bitstream, syntax structure(s) that contain the coded region as well as a different syntax structure that indicates the completion of the coded region. Another media processing tool such as a decoder can receive, in a predefined order in an elementary bitstream, syntax structure(s) that contain a coded region of a picture as well as a different syntax structure that indicates the completion of the coded region.

Abstract: To decode encoded video using a computer with a central processing unit and a graphics processing unit as a coprocessor, parameters applied to blocks of intermediate image data are transferred from the central processing unit to the graphics processing unit. When the operation being performed applies to a small portion of the blocks of intermediate image data, then the central processing unit can transfer to the graphics processing unit the parameters for only those blocks to which the operation applies. In particular, the central processing unit can transfer a set of parameters for a limited number of blocks of intermediate image data, with an indication of the block to which each set of parameters applies, which both can improve speed of operation and can reduce power consumption.

Abstract: Innovations in the area of sample metadata processing can help a media playback tool avoid loss of synchronization between sample metadata and media samples. For example, a media playback tool identifies encoded data and sample metadata for a current media sample, then couples the sample metadata with the current media sample. The media playback tool provides the sample metadata and encoded data for the current media sample to a media decoder, which maintains the coupling between at least one element of the sample metadata and the current media sample during at least one stage of decoding, even when the current media sample is dropped, delayed, split, or repeated. For example, the media playback tool can determine whether to drop the current media sample and, if the current media sample is dropped, also drop the sample metadata that is coupled with the current media sample.