Abstract: The present disclosure provides devices and techniques for processing a media capture stream captured by a camera device using a chain device media foundation transform (DMFT). The techniques include configuring multiple DMFTs such that an original manufacturer (OEM) may have flexibility in independently selecting various functionalities from different sources (e.g., OS, OEM, IHV, ISV, or VARs) in order to maximize hardware capabilities while minimizing the drawbacks of creating a single DMFT. To that end, the implementation of the present disclosure includes a devices and techniques of chainable DMFTs such that a device transform manager may select a set of functionalities (e.g., face recognition, color effects, etc.) from multiple vendors to customize the camera's capabilities according to the OEM specification.

Abstract: Methods and devices for enabling controls of an imaging device may include receiving a data stream with a request for at least one control of an imaging device, wherein the at least one control is related to Universal Serial Bus (USB) video. The methods and devices may include identifying a defined functionality of the at least one control and transmitting the data stream to the imaging device to specify the operation of the imaging device based on the defined functionality of the at least one control. The methods and devices may also include generating metadata information from received video frame packets from an imaging device. The methods and devices may include identifying metadata information in the header of a video packet when the header size exceeds the standard header size. The methods and devices may include generating a metadata buffer with the identified metadata information.

Abstract: Error detecting and protection innovations for video decoders are described. For example, in a multithreaded video decoder, a picture extent discovery (PED) task detects an error in a video bitstream which corrupts a picture. The PED task then determines any PED sub-stage which have been completed for the picture, and based on this determination, performs error-handing PED operations. In another example, an entropy decoding (ED) task checks validity on a macroblock-by-macroblock basis using a redundant buffer to avoid overflows. Additionally, error recovery innovations are described which facilitate playback of a video bit stream at an arbitrary position. For example, a video decoder chooses a picture in the bit stream after the arbitrary position at which to begin decoding based on a determination of acceptable recovery time and/or acceptable picture quality.

Abstract: Techniques for combining images to reduce motion blur in an output image are described. Data indicative of a first image and a second image of a scene is received. At least one portion of the first image associated with motion blur is identified. The portion of the first image is compared to a corresponding second portion of the second image. First pixels within the at least one portion of the first image and second pixels within the second portion of the second image not associated with motion blur are selected. The selected first pixels and second pixels are combined into an output image. The combined selected pixels are rendered on a display device.

Abstract: A decoder which can detect errors in MPEG-2 coefficient blocks can identify syntactically-correct blocks which have out-of-bounds coefficients. The decoder computes coefficient bounds based on quantization scalers and quantization matrices and compares these to coefficient blocks during decoding; if a block has out-of-bounds coefficients, concealment is performed on the block. In a decoder implemented all in software, coefficient bounds checking is performed on iDCT coefficients against upper and lower bounds in a spatial domain. In a decoder which performs iDCT in hardware, DCT coefficients are compared to an upper energy bound.

Abstract: Reduced latency video stabilization methods and tools generate truncated filters for use in the temporal smoothing of global motion transforms representing jittery motion in captured video. The truncated filters comprise future and past tap counts that can be different from each other and are typically less than those of a baseline filter providing a baseline of video stabilization quality. The truncated filter future tap count can be determined experimentally by comparing a smoothed global motion transform set generated by applying a baseline filter to a video segment to those generated by multiple test filter with varying future tap counts, then settings the truncated filter future tap count based on an inflection point on an error-future tap count curve. A similar approach can be used to determine the truncated filter past tap count.

Abstract: A video decoder is disclosed that uses metadata in order to make optimization decisions. In one embodiment, metadata is used to choose which of multiple available decoder engines should receive a video sequence. In another embodiment, the optimization decisions can be based on length and location metadata information associated with a video sequence. Using such metadata information, a decoder engine can skip start-code scanning to make the decoding process more efficient. Also based on the choice of decoder engine, it can decide whether emulation prevention byte removal shall happen together with start code scanning or not.

Abstract: Error concealment techniques for video decoding are described. For example, a video decoder after finding a corrupted picture in a bit stream, finds a suitable neighbor for the corrupted picture. For example, the video decoder favors pictures with the same parity as the corrupted picture and considers picture order count and picture corruption in choosing a neighbor. The decoder then modifies syntax elements for the encoded video in the bit stream to allow the neighbor to be used in concealing the corruption in the corrupted picture. The modification of syntax elements can depend on the particular video decoder implementation. For example, in a software-only multithreaded video decoder, a task graph is modified, while in a system utilizing video acceleration, syntax elements for reference lists are modified.

Abstract: One or more techniques and/or systems are provided for camera capture recommendation. For example, an application may operate to capture an image using a capture device (e.g., a user may use a camera of a smart phone to capture a vacation photo for sharing through a social network app). Camera parameters of the capture device and/or a preview data stream (e.g., pixel data depicting a beach “seen” by the camera in real-time) may be used to generate a camera capture recommendation (e.g., a recommendation to use a haze removal module, a high dynamic range module, a focus bracketing module, etc.). The camera capture recommendation is provided to the application. In this way, the application may selectively use, override, supplement (e.g., use an application supplied module), or modify the camera capture recommendation for application to the capture device to obtain an output image.

Abstract: Video image stabilization provides better performance on a generic platform for computing devices by evaluating available multimedia digital signal processing components, and selecting the available components to utilize according to a hierarchy structure for video stabilization performance for processing parts of the video stabilization. The video stabilization has improved motion vector estimation that employs refinement motion vector searching according to a pyramid block structure relationship starting from a downsampled resolution version of the video frames. The video stabilization also improves global motion transform estimation by performing a random sample consensus approach for processing the local motion vectors, and selection criteria for motion vector reliability. The video stabilization achieves the removal of hand shakiness smoothly by real-time one-pass or off-line two-pass temporal smoothing with error detection and correction.

Abstract: A battery operated device, having a display with two or more available refresh rates, has its refresh rate selected so as to match the video frame rate of video data played back on the display. This selection is made by coordinating the resources in the device that are used to process the video from its reception through to its display.

Abstract: Video image stabilization provides better performance on a generic platform for computing devices by evaluating available multimedia digital signal processing components, and selecting the available components to utilize according to a hierarchy structure for video stabilization performance for processing parts of the video stabilization. The video stabilization has improved motion vector estimation that employs refinement motion vector searching according to a pyramid block structure relationship starting from a downsampled resolution version of the video frames. The video stabilization also improves global motion transform estimation by performing a random sample consensus approach for processing the local motion vectors, and selection criteria for motion vector reliability. The video stabilization achieves the removal of hand shakiness smoothly by real-time one-pass or off-line two-pass temporal smoothing with error detection and correction.

Abstract: A media processing tool adds custom data to an elementary media bitstream or media container. The custom data indicates nominal range of samples of media content, but the meaning of the custom data is not defined in the codec format or media container format. For example, the custom data indicates the nominal range is full range or limited range. For playback, a media processing tool parses the custom data and determines an indication of media content type. A rendering engine performs color conversion operations whose logic changes based at least in part on the media content type. In this way, a codec format or media container format can in effect be extended to support full nominal range media content as well as limited nominal range media content, and hence preserve full or correct color fidelity, while maintaining backward compatibility and conformance with the codec format or media container format.

Abstract: A media processing tool adds custom data to an elementary media bitstream or media container. The custom data indicates nominal range of samples of media content, but the meaning of the custom data is not defined in the codec format or media container format. For example, the custom data indicates the nominal range is full range or limited range. For playback, a media processing tool parses the custom data and determines an indication of media content type. A rendering engine performs color conversion operations whose logic changes based at least in part on the media content type. In this way, a codec format or media container format can in effect be extended to support full nominal range media content as well as limited nominal range media content, and hence preserve full or correct color fidelity, while maintaining backward compatibility and conformance with the codec format or media container format.

Abstract: A facility for generating at least one image is described. For each of multiple registered photography scenarios, the facility determines a suitable score for the scenario based upon state of a photography device. The facility selects a scenario having a suitability score that is no lower than any other determined suitability score. The facility then captures a sequence of one or more frames in a manner specified for the selected scenario, and processes that captured sequence of frames in a manner specified for the selected scenario to obtain at least one image.

Abstract: An audio encoder implements multi-channel coding decision, band truncation, multi-channel rematrixing, and header reduction techniques to improve quality and coding efficiency. In the multi-channel coding decision technique, the audio encoder dynamically selects between joint and independent coding of a multi-channel audio signal via an open-loop decision based upon (a) energy separation between the coding channels, and (b) the disparity between excitation patterns of the separate input channels. In the band truncation technique, the audio encoder performs open-loop band truncation at a cut-off frequency based on a target perceptual quality measure. In multi-channel rematrixing technique, the audio encoder suppresses certain coefficients of a difference channel by scaling according to a scale factor, which is based on current average levels of perceptual quality, current rate control buffer fullness, coding mode, and the amount of channel separation in the source.

Abstract: A video decoder is disclosed that uses metadata in order to make optimization decisions. In one embodiment, metadata is used to choose which of multiple available decoder engines should receive a video sequence. In another embodiment, the optimization decisions can be based on length and location metadata information associated with a video sequence. Using such metadata information, a decoder engine can skip start-code scanning to make the decoding process more efficient. Also based on the choice of decoder engine, it can decide whether emulation prevention byte removal shall happen together with start code scanning or not.

Abstract: One or more techniques and/or systems are provided for video stabilization and/or for image frame generation. For example, a user may instruct a video application hosted on a smart phone to capture a video at a target resolution of 1080 pixels. A padded input having a padded resolution that is larger than the target resolution may be obtained from a capture device, such as a camera of the smart phone. The padded input may be provided to a video stabilization component to obtain a target image frame having the target resolution. In this way, the video stabilization component may perform cropping using padded margin pixels (e.g., additional pixels of the padded input beyond the 1080 pixels of the target resolution) so that image upscaling after cropping (e.g., to account for global warping, etc.) may be mitigated to reduce blur that may otherwise result from image upscaling.

Abstract: One or more techniques and/or systems are provided for video stabilization and/or for image frame generation. For example, a user may instruct a video application hosted on a smart phone to capture a video at a target resolution of 1080 pixels. A padded input having a padded resolution that is larger than the target resolution may be obtained from a capture device, such as a camera of the smart phone. The padded input may be provided to a video stabilization component to obtain a target image frame having the target resolution. In this way, the video stabilization component may perform cropping using padded margin pixels (e.g., additional pixels of the padded input beyond the 1080 pixels of the target resolution) so that image upscaling after cropping (e.g., to account for global warping, etc.) may be mitigated to reduce blur that may otherwise result from image upscaling.

Abstract: One or more techniques and/or systems are provided for camera capture recommendation. For example, an application may operate to capture an image using a capture device (e.g., a user may use a camera of a smart phone to capture a vacation photo for sharing through a social network app). Camera parameters of the capture device and/or a preview data stream (e.g., pixel data depicting a beach “seen” by the camera in real-time) may be used to generate a camera capture recommendation (e.g., a recommendation to use a haze removal module, a high dynamic range module, a focus bracketing module, etc.). The camera capture recommendation is provided to the application. In this way, the application may selectively use, override, supplement (e.g., use an application supplied module), or modify the camera capture recommendation for application to the capture device to obtain an output image.