Abstract: Systems and methods estimate occluded pixels in frames of a video sequence. Optical flow data is received to determine a validity for forward and backward flow vectors for a common pixel location in a first frame and a second frame that are temporally next to one another. Occlusion information for the first frame determines pixels that are hidden in the second frame with respect to playback from the first frame to the second frame. Occlusion information for the second frame determines pixels that are hidden in the first frame with respect to playback from the second frame to the first frame.

Abstract: Systems and methods estimate optical flow vectors for occluded pixels between frames of a video sequence. Regions of occluded pixels may be identified and a cause of their occlusion may be determined. Different estimation techniques may be applied based, at least in part, on the cause of occlusion to provide a lightweight, less resource intensive estimation of optical flow data. Optical flow vectors for pixels that are occluded due to movement out of a frame may be estimated using a first technique while optical flow vectors for pixels that are occluded due to foreground movement may be estimated using a second technique.

Abstract: Systems and methods herein address reference frame selection in video streaming applications using one or more processing units to decode a frame of an encoded video stream that uses an inter-frame depicting an object and an intra-frame depicting the object, the intra-frame being included in a set of intra-frames based at least in part on at least one attribute of the object as depicted in the intra-frame being different from the at least one attribute of the object as depicted in other intra-frames of the set of intra-frames.

Abstract: Systems and methods relate to facial video encoding and reconstruction, particularly in ultra-low bandwidth settings. In embodiments, a video conferencing or other streaming application uses automatically tracked feature cropping information. A bounding shape size—used to identify the cropped region—varies and is dynamically determined to maintain a proportion for feature reconstruction, such as resizing in the event of a zoom-in on a face (or other feature of interest) or a zoom-out. The tracking scheme may be used to smooth sudden movements, including lateral ones, to generate more natural transitions between frames. Tracking and cropping information (e.g., size and position of the cropped region) may be embedded within an encoded bitstream as supplemental enhancement information (“SEI”), for eventual decoding by a receiver and for compositing a decoded face at a proper location in the applicable stream.

Abstract: Systems and methods herein address reference frame selection in video streaming applications using one or more processing units to replace, during receipt of an encoded video stream, a first set of frames stored in a cache with a second set of frames based at least in part on an indication within the encoded video stream that the second set of frames includes a non-blurred frame (NBF).

Abstract: Systems and methods herein address reference frame selection in video streaming applications using one or more processing units to identify a frame of a sequence of frames as a blurred frame based at least in part on a first variance of motion (VoM) of the frame being less than or equal to an adaptive threshold that is based in part on a moving average of variance of motion (MAoV) determined using one or more reference frames.

Abstract: In examples, when attempting to interpolate or extrapolate a frame based on motion vectors of two adjacent frames, there can be more than one pixel value mapped to a given location in the frame. To select between conflicting pixel values for the given location, similarities between the motion vectors of source pixels that cause the conflict and global flow may be evaluated. For example, a level of similarity for a motion vector may be computed using a similarity metric based at least on a difference between an angle of a global motion vector and an angle of the motion vector. The similarity metric may also be based at least on a difference between a magnitude of the global motion vector and a magnitude of the motion vector. The similarity metric may weigh the difference between the angles in proportion to the magnitude of the global motion vector.

Abstract: Apparatuses, systems, and techniques are presented to track objects represented in images or video data. In at least one embodiment, motion of one or more objects within a plurality of digital images is determined based, at least in part, on flow information corresponding to the one or more objects.

Abstract: Apparatuses, systems, and techniques are presented to track objects represented in images or video data. In at least one embodiment, motion information corresponding to one or more objects within one or more digital images is used to determine a representative set of motion information corresponding to the motion information for the one or more objects.