Abstract: A system for delivering live streaming content based on accurate media data fragment size and duration. The system may include a client media player to receive a portion of a streaming media file (e.g., in an MP4 format), download a first sub-portion of the streaming media file including fragment-level metadata, and parse and analyze the fragment-level metadata to determine a size and duration of a current fragment of the media file. A media server may generate custom data identifying a size and duration of a current fragment of a media file. The media server may insert the custom data (e.g., as a custom header or unique packet identifier) and send the custom data to a client media player. The client media player may be configured to decode the custom data and determine the current fragment size and duration.

Abstract: Video image stabilization provides better performance on a generic platform for computing devices by evaluating available multimedia digital signal processing components, and selecting the available components to utilize according to a hierarchy structure for video stabilization performance for processing parts of the video stabilization. The video stabilization has improved motion vector estimation that employs refinement motion vector searching according to a pyramid block structure relationship starting from a downsampled resolution version of the video frames. The video stabilization also improves global motion transform estimation by performing a random sample consensus approach for processing the local motion vectors, and selection criteria for motion vector reliability. The video stabilization achieves the removal of hand shakiness smoothly by real-time one-pass or off-line two-pass temporal smoothing with error detection and correction.

Abstract: Techniques are described for encoding noisy media content to improve its visual quality. Quantization parameters can be applied to residual coefficient matrices of portions of an image frame as a compression technique. To improve the visual quality of media content, the quantization parameters to be applied can be adjusted to fit within a range around a representative quantization parameter of all of the portions of the image frame.

Abstract: Techniques are described for decoding portions of image frames for virtual reality (VR) media content. A field of view of a viewer within a VR environment can be determined and used to decode a portion of an image frame that provides image content for that portion. The other portions of the image frame that are not within the image frame can remain non-decoded when not in the visible field of view.

Abstract: Techniques are described that enable virtual reality content to be delivered. These techniques include decoding reference frames of video content for non-viewed sections of the video content.

Abstract: This application relates to video encoding and decoding, and specifically to tools and techniques for using and providing supplemental enhancement information in bitstreams. Among other things, the detailed description presents innovations for bitstreams having supplemental enhancement information (SEI). In particular embodiments, the SEI message includes picture source data (e.g., data indicating whether the associated picture is a progressive scan picture or an interlaced scan picture and/or data indicating whether the associated picture is a duplicate picture). The SEI message can also express a confidence level of the encoder's relative confidence in the accuracy of this picture source data. A decoder can use the confidence level indication to determine whether the decoder should separately identify the picture as progressive or interlaced and/or a duplicate picture or honor the picture source scanning information in the SEI as it is.

Abstract: Techniques are described by which multiple, independently encoded video streams may be combined into a single decodable video stream. These techniques take advantage of existing features of commonly used video codecs that support the independent encoding of different regions of an image frame (e.g., H.264 slices or HEVC tiles). Instead of including different parts of the same image, each region corresponds to the encoded image data of the frames of one of the independent video streams.

Abstract: Innovations in encoder-side options for intra block copy (“BC”) prediction mode facilitate intra BC prediction that is more effective in terms of rate-distortion performance and/or computational efficiency of encoding. For example, some of the innovations relate to estimation of sample values within an overlap area of a current block during block vector estimation. Other innovations relate to prediction of block vector (“BV”) values during encoding or decoding using “ping-pong” approaches.

Abstract: Techniques are described for communicating encoded data using start code emulation prevention. The described techniques include obtaining at least one partially encrypted packet, identifying at least one portion of the packet that is unencrypted, and determining that the identified unencrypted portion(s) emulates a start code. Start code emulation prevention data or emulation prevention bytes (EPBs) may be inserted into only the encrypted portion of the packet. The modified packet may be communicated to another device/storage, along with an indication of which portion(s) of the packet are unencrypted. Upon receiving the packet and indication, the receiving device may identify and remove the EPBs in the identified unencrypted portion(s) of the packet, and decrypt the packet to recover the data. In some aspects, upon identifying the indication, the receiving device may only search for EPBs in the unencrypted portion(s) of the packet, thus yielding a more efficient start code emulation prevention process.

Abstract: Methods and apparatus are described for partitioning and reordering block-based matrix multiplications for high-speed data streaming in general matrix multiplication (GEMM), which may be implemented by a programmable integrated circuit (IC). By preloading and hierarchically caching the blocks, examples of the present disclosure reduce the double data rate (DDR) memory intake bandwidth for software-defined GEMM accelerators.

Abstract: Techniques are described for encoding image frames of media content to be displayed within a three-dimensional projection space. Characteristics of the image frame when transformed to fit within the projection space can be determined and used to generate encoding parameters. The image frame can then be encoded using those encoding parameters, and therefore, improve the playback of the media content when it is displayed within the projection space.

Abstract: Video content is protected using a digital rights management (DRM) mechanism, the video content having been previously encrypted and compressed for distribution, and also including metadata such as closed captioning data, which might be encrypted or clear. The video content is obtained by a system of a computing device, the metadata is extracted from the video content and provided to a video decoder, and the video content is provided to a secure DRM component. The secure DRM component decrypts the video content and provides the decrypted video content to a secure decoder component of a video decoder. As part of the decryption, the secure DRM component drops the metadata that was included in the obtained video content. However, the video decoder receives the extracted metadata in a non-protected environment and thus is able to provide the extracted metadata and the decoded video content to a content playback application.

Abstract: Techniques are described for synchronizing audio content and video content when server-side fragment insertion techniques are used.

Abstract: Techniques are described that enable the use of variable bit rate (VBR) encoding for live content.

Abstract: Syntax structures that indicate the completion of coded regions of pictures are described. For example, a syntax structure in an elementary bitstream indicates the completion of a coded region of a picture. The syntax structure can be a type of network abstraction layer unit, a type of supplemental enhancement information message or another syntax structure. For example, a media processing tool such as an encoder can detect completion of a coded region of a picture, then output, in a predefined order in an elementary bitstream, syntax structure(s) that contain the coded region as well as a different syntax structure that indicates the completion of the coded region. Another media processing tool such as a decoder can receive, in a predefined order in an elementary bitstream, syntax structure(s) that contain a coded region of a picture as well as a different syntax structure that indicates the completion of the coded region.

Abstract: Innovations in the area of sample metadata processing can help a media playback tool avoid loss of synchronization between sample metadata and media samples. For example, a media playback tool identifies encoded data and sample metadata for a current media sample, then couples the sample metadata with the current media sample. The media playback tool provides the sample metadata and encoded data for the current media sample to a media decoder, which maintains the coupling between at least one element of the sample metadata and the current media sample during at least one stage of decoding, even when the current media sample is dropped, delayed, split, or repeated. For example, the media playback tool can determine whether to drop the current media sample and, if the current media sample is dropped, also drop the sample metadata that is coupled with the current media sample.

Abstract: Buffer optimization techniques are described herein in which a graphics processing system is configured to implement and select between a plurality of buffer schemes for processing of an encoded data stream in dependence upon formats used for decoding and rendering (e.g., video format, bit depth, resolution, content type, etc.) and device capabilities such as available memory and/or processing power. Processing of an encoded data stream for display and rendering via the graphics processing system then occurs using a selected one of the buffer schemes to define buffers employed for the decoding and rendering, including at least configuring the sizes of buffers. The plurality of schemes may include at least one buffer scheme for processing the encoded content when the input format and the output format are the same, and a different buffer scheme for processing the encoded content when the input format and the output format are different.

Abstract: Methods and apparatus are described for stitching independently encoded content streams together such that they appear to be part of a single streaming session.

Abstract: Embodiments herein describe techniques for static scheduling a neural network implemented in a massively parallel hardware system. The neural network may be scheduled using three different scheduling levels referred to herein as an upper level, an intermediate level, and a lower level. In one embodiment, the upper level includes a hardware or software model of the layers in the neural network that establishes a sequential order of functions that operate concurrently in the hardware system. In the intermediate level, identical processes in the functions defined in the upper level are connected to form a systolic array or mesh and balanced data flow channels are used to minimize latency. In the lower level, a compiler can assign the operations performed by the processing elements in the systolic array to different portions of the hardware system to provide a static schedule for the neural network.

Abstract: In the disclosed methods and systems for processing in a neural network system, a host computer system writes a plurality of weight matrices associated with a plurality of layers of a neural network to a memory shared with a neural network accelerator. The host computer system further assembles a plurality of per-layer instructions into an instruction package. Each per-layer instruction specifies processing of a respective layer of the plurality of layers of the neural network, and respective offsets of weight matrices in a shared memory. The host computer system writes input data and the instruction package to the shared memory. The neural network accelerator reads the instruction package from the shared memory and processes the plurality of per-layer instructions of the instruction package.