Abstract: A technique for streaming and a client device that uses the technique are disclosed herein. The disclosed technique determines context complexity of streamed data and determines whether to discard or select the streamed data for a future reference frame based on the context complexity of the streamed data. The streamed data is discarded if the content complexity is higher than a content complexity threshold, and the streamed data is selected if the content complexity is not higher than a content complexity threshold. This is based on the realization that error propagation in the case of a less complex video sequence is not very bothersome to the end user experience whereas corruption will be very severe in cases of highly complex sequences.

Abstract: Systems and methods for efficient lossless compression of captured raw image information are presented. A method can comprise: receiving raw image data from an image capture device, segregating the pixel data into a base layer portion and an enhanced layer portion, reconfiguring the base layer portion expressed in the first color space values from a raw capture format into a pseudo second color space compression mechanism compatible format, and compressing the reconfigured base layer portion of first color space values. The raw image data can include pixel data are expressed in first color space values. The segregation can be based upon various factors, including a compression benefits analysis of a boundary location between the base layer portion and enhanced layer portion. The reconfiguring the base layer portion can include separating the base layer portion based upon multiple components within the raw data; and forming base layer video frames from the multiple components.

Abstract: In various examples, a deep neural network (DNN) based pre-filter for content streaming applications is used to dynamically adapt scene entropy (e.g., complexity) in response to changing network or system conditions of an end-user device. For example, where network and/or system performance issues or degradation are identified, the DNN may be implemented as a frame pre-filter to reduce the complexity or entropy of the frame prior to streaming—thereby allowing the frame to be streamed at a reduced bit rate without requiring a change in resolution. The DNN-based pre-filter may be tuned to maintain image detail along object, boundary, and/or surface edges such that scene navigation—such as by a user participating in an instance of an application—may be easier and more natural to the user.

Abstract: In various examples, machine learning of encoding parameter values for a network is performed using a video encoder. Feedback associated with streaming video encoded by a video encoder over a network may be applied to an MLM(s). Using such feedback, the MLM(s) may predict a value(s) of an encoding parameter(s). The video encoder may then use the value to encode subsequent video data for the streaming. By using the video encoder in training, the MLM(s) may learn based on actual encoded parameter values of the video encoder. The MLM(s) may be trained via reinforcement learning based on video encoded by the video encoder. A rewards metric(s) may be used to train the MLM(s) using data generated or applied to the physical network in which the MLM(s) is to be deployed and/or a simulation thereof. Penalty metric(s) (e.g., the quantity of dropped frames) may also be used to train the MLM(s).

Abstract: Systems and methods for efficient lossless compression of captured raw image information are presented. A method can comprise: receiving raw image data from an image capture device, segregating the pixel data into a base layer portion and an enhanced layer portion, reconfiguring the base layer portion expressed in the first color space values from a raw capture format into a pseudo second color space compression mechanism compatible format, and compressing the reconfigured base layer portion of first color space values. The raw image data can include pixel data are expressed in first color space values. The segregation can be based upon various factors, including a compression benefits analysis of a boundary location between the base layer portion and enhanced layer portion. The reconfiguring the base layer portion can include separating the base layer portion based upon multiple components within the raw data; and forming base layer video frames from the multiple components.

Abstract: Systems and methods for efficient lossless compression of captured raw image information are presented. A method can comprise: receiving raw image data from an image capture device, segregating the pixel data into a base layer portion and an enhanced layer portion, reconfiguring the base layer portion expressed in the first color space values from a raw capture format into a pseudo second color space compression mechanism compatible format, and compressing the reconfigured base layer portion of first color space values. The raw image data can include pixel data are expressed in first color space values. The segregation can be based upon various factors, including a compression benefits analysis of a boundary location between the base layer portion and enhanced layer portion. The reconfiguring the base layer portion can include separating the base layer portion based upon multiple components within the raw data; and forming base layer video frames from the multiple components.

Abstract: A technique for streaming and a client device that uses the technique are disclosed herein. The disclosed technique determines context complexity of streamed data and determines whether to discard or select the streamed data for a future reference frame based on the context complexity of the streamed data. The streamed data is discarded if the content complexity is higher than a content complexity threshold, and the streamed data is selected if the content complexity is not higher than a content complexity threshold. This is based on the realization that error propagation in the case of a less complex video sequence is not very bothersome to the end user experience whereas corruption will be very severe in cases of highly complex sequences.

Abstract: A viewing device, a method of displaying streamed data frames and a client viewing device are disclosed herein. In one embodiment, the video viewing device includes: (1) a screen, (2) a decoder configured to decode a data frame received in a bitstream from a transmitter to provide a decoded data frame, and (3) an error concealer configured to either discard the decoded data frame or select the decoded data frame for display on the screen based on a complexity of the decoded data frame.

Abstract: Systems and methods for efficient lossless compression of captured raw image information are presented. A method can comprise: receiving raw image data from an image capture device, segregating the pixel data into a base layer portion and an enhanced layer portion, reconfiguring the base layer portion expressed in the first color space values from a raw capture format into a pseudo second color space compression mechanism compatible format, and compressing the reconfigured base layer portion of first color space values. The raw image data can include pixel data are expressed in first color space values. The segregation can be based upon various factors, including a compression benefits analysis of a boundary location between the base layer portion and enhanced layer portion. The reconfiguring the base layer portion can include separating the base layer portion based upon multiple components within the raw data; and forming base layer video frames from the multiple components.

Abstract: A viewing device, a method of displaying streamed data frames and a client viewing device are disclosed herein. In one embodiment, the video viewing device includes: (1) a screen, (2) a decoder configured to decode a data frame received in a bitstream from a transmitter to provide a decoded data frame, and (3) an error concealer configured to either discard the decoded data frame or select the decoded data frame for display on the screen based on a complexity of the decoded data frame.

Abstract: An enhanced display encoder system for a video stream source includes an enhanced video encoder that has parallel intra frame and inter frame encoding units for encoding a video frame, wherein an initial number of macroblocks is encoded to determine a scene change status of the video frame. Additionally, a video frame history unit determines an intra frame update status for the video frame from a past number of video frames, and an encoder selection unit selects the intra frame or inter frame encoding unit for further encoding of the video frame to support a wireless transmission based on the scene change status and the intra frame update status. A method of enhanced video frame encoding for video stream sourcing is also provided.

Abstract: A technique of encoding video frames allocates an available number of bits to different portions of the video frame. A processing unit identifies a region of interest (ROI) in a video frame, and computes a first and second complexity parameter respectively representing the change in video information in the ROI portions and non-ROI portions in the video frame relative to a reference frame. Bits are allocated to the ROI portion proportional (positive correlation) to the first complexity parameter and a ratio of the area of the ROI to the area of the frame. The remaining available bits are allocated to the non-ROI. In an embodiment, the bits are encoded according to H.264 standard.

Abstract: An image processor in an image capture device compensates for the effects of undesirable camera shakes occurring during video capture The image processor receives a pair of source frames representing images of a scene, generates a pair of subsampled frames from the source frames, and computes a coarse displacement of the captured image due to camera shakes by comparing the two subsampled frames. The image processor may then refine the determined coarse displacement by comparing the two source frames and a bound determined by an extent of subsampling, and compensate for the displacement accordingly. Display aberrations such as blank spaces caused due to shifting are also avoided by displaying only a portion of the captured image and shifting the displayed portion to compensate for camera shake. The image processor also recognizes displacements due to intentional camera movement, and does not correct for such displacements.

Abstract: Multiple sets of pixel values representing a captured image of a scene are received, with each set representing an image captured with a corresponding degree of focus. An image processor may identify a region of interest in the captured image, automatically determine the configuration parameters for a lens assembly to provide a desired degree of focus for the region of interest, and generate signals to configure a lens assembly. In an embodiment, the region of interest is a face, the desired degree of focus of the face is determined by computing a rate of variation of luminance of pixels representing the face, and the desired degree is the degree of the image having the maximum degree of focus.

Abstract: Multiple sets of pixel values representing a captured image of a scene are received, with each set representing an image captured with a corresponding degree of focus. An image processor may identify a region of interest in the captured image, automatically determine the configuration parameters for a lens assembly to provide a desired degree of focus for the region of interest, and generate signals to configure a lens assembly. In an embodiment, the region of interest is a face, the desired degree of focus of the face is determined by computing a rate of variation of luminance of pixels representing the face, and the desired degree is the degree of the image having the maximum degree of focus.

Abstract: An image processor, which determines appropriate exposure parameters for a shutter assembly in a camera. The image processor may computationally determine a region of interest in a scene sought to be captured, and set the parameters to ensure that the exposure parameters are set to capture an image of the scene with the region of interest having a desired brightness level. In an embodiment, pixel values of multiple frames (each frame with a corresponding set of configuration parameters of the shutter assembly) may be examined to determine the frame having pixel values with the region having the desired brightness level. The shutter assembly may be configured with the parameters corresponding to such a frame to provide an auto-exposure feature.

Abstract: Multiple sets of pixel values representing a captured image of a scene are received, with each set representing an image captured with a corresponding degree of focus. An image processor may identify a region of interest in the captured image, automatically determine the configuration parameters for a lens assembly to provide a desired degree of focus for the region of interest, and generate signals to configure a lens assembly. In an embodiment, the region of interest is a face, the desired degree of focus of the face is determined by computing a rate of variation of luminance of pixels representing the face, and the desired degree is the degree of the image having the maximum degree of focus.

Abstract: An image processor in an image capture device compensates for the effects of undesirable camera shakes occurring during video capture The image processor receives a pair of source frames representing images of a scene, generates a pair of subsampled frames from the source frames, and computes a coarse displacement of the captured image due to camera shakes by comparing the two subsampled frames. The image processor may then refine the determined coarse displacement by comparing the two source frames and a bound determined by an extent of subsampling, and compensate for the displacement accordingly. Display aberrations such as blank spaces caused due to shifting are also avoided by displaying only a portion of the captured image and shifting the displayed portion to compensate for camera shake. The image processor also recognizes displacements due to intentional camera movement, and does not correct for such displacements.

Abstract: A technique of encoding video frames allocates an available number of bits to different portions of the video frame. A processing unit identifies a region of interest (ROI) in a video frame, and computes a first and second complexity parameter respectively representing the change in video information in the ROI portions and non-ROI portions in the video frame relative to a reference frame. Bits are allocated to the ROI portion proportional (positive correlation) to the first complexity parameter and a ratio of the area of the ROI to the area of the frame. The remaining available bits are allocated to the non-ROI. In an embodiment, the bits are encoded according to H.264 standard.

Abstract: Multiple sets of pixel values representing a captured image of a scene are received, with each set representing an image captured with a corresponding degree of focus. An image processor may identify a region of interest in the captured image, automatically determine the configuration parameters for a lens assembly to provide a desired degree of focus for the region of interest, and generate signals to configure a lens assembly. In an embodiment, the region of interest is a face, the desired degree of focus of the face is determined by computing a rate of variation of luminance of pixels representing the face, and the desired degree is the degree of the image having the maximum degree of focus.