Abstract: Systems and techniques are described herein for processing video data. For instance, a technique can include receiving a first image after a previous image. The process can further include receiving a first segmentation mask associated with the previous image. The process can also include estimating a first set of forward motion vectors between the previous image and the first image. The process can further include estimating a reliability of the first set of forward motion vectors. The process can also include extrapolating a second segmentation mask associated with the first image using the first set of forward motion vectors and first segmentation mask based on the estimated reliability of the first set of forward motion vectors.

Abstract: A device includes a memory and one or more processors. The memory is configured to store an image enhancement network of an image enhancer. The one or more processors are configured to predict an image compression quality of an image of a stream of images. The one or more processors are also configured to configure the image enhancer based on the image compression quality. The one or more processors are further configured to process, using the image enhancement network of the configured image enhancer, the image to generate an enhanced image.

Abstract: A device includes a memory and one or more processors. The memory is configured to store instructions. The one or more processors are configured to execute the instructions to apply a neural network to a first image to generate an enhanced image. The one or more processors are also configured to execute the instructions to adjust at least a portion of a high-frequency component of the enhanced image based on a motion compensation operation to generate an adjusted high-frequency image component. The one or more processors are further configured to execute the instructions to combine a low-frequency component of the enhanced image and the adjusted high-frequency image component to generate an adjusted enhanced image.

Abstract: A device includes one or more processors configured to execute instructions to obtain motion data indicating estimated motion between a first frame and a second frame of an input sequence of image frames, and to identify, based on the motion data, any frame regions of the first frame that indicate motion greater than a motion threshold. The one or more processors are also configured to determine, based on the motion data, a motion metric associated with the identified frame regions, and to perform a determination, based on the motion metric and a size metric associated with the identified frame regions, whether to use motion-compensated frame interpolation to generate an intermediate frame. The one or more processors are further configured to generate the intermediate frame based on the determination, and to generate an output sequence of image frames that includes the intermediate frame between the first frame and the second frame.

Abstract: A device includes a memory configured to store an adapted network that is configured to generate a modified image based on a single image. The device includes a processor configured to obtain, from a stream of video data, a first distorted image depicting an object, and to provide the first distorted image to the adapted network to generate a first modified image. The processor is configured to obtain, from the stream of video data, a second distorted image depicting the object, and to provide the second distorted image to the adapted network to generate a second modified image. The object is distorted differently in the second distorted image than in the first distorted image. The processor is configured to generate a video output including the first modified image and the second modified image without visible artifacts due to distortion differences between the first distorted image and the second distorted image.

Abstract: A device includes one or more processors configured to execute instructions to obtain motion data indicating estimated motion between a first frame and a second frame of an input sequence of image frames, and to identify, based on the motion data, any frame regions of the first frame that indicate motion greater than a motion threshold. The one or more processors are also configured to determine, based on the motion data, a motion metric associated with the identified frame regions, and to perform a determination, based on the motion metric and a size metric associated with the identified frame regions, whether to use motion-compensated frame interpolation to generate an intermediate frame. The one or more processors are further configured to generate the intermediate frame based on the determination, and to generate an output sequence of image frames that includes the intermediate frame between the first frame and the second frame.

Abstract: Technologies are provided for processing data in neural networks. An example method can include processing, by each layer of a neural network, a row in a first stripe of a data input, the row being processed sequentially in a horizontal direction and according to a layer-by-layer sequence; after processing the row, processing, by each layer, subsequent rows in the first stripe on a row-by-row basis, each subsequent row being processed sequentially in the horizontal direction and according to the layer-by-layer sequence; generating an output stripe based on the processing of the row and subsequent rows; processing, by each layer, a second stripe of the data input, each row in the second stripe being processed in the horizontal direction and according to the layer-by-layer sequence, wherein rows in the second stripe are processed on a row-by-row basis; and generating another output stripe based on the processing of the second stripe.

Abstract: A camera image processor receives frames from an image sensor of the camera. While continuing to receive frames from the image sensor, the camera image processor detecting an action within one or more frames of the received frames, for example using a convolution neural network trained to recognize one or more actions. While continuing to receive frames from the image sensor, the camera image processor captures the one or more frames with the detected action, for example as a still image, or as a slow-motion portion of a video that includes the received frames.

Abstract: A device includes a memory configured to store an adapted network that is configured to generate a modified image based on a single image. The device includes a processor configured to obtain, from a stream of video data, a first distorted image depicting an object, and to provide the first distorted image to the adapted network to generate a first modified image. The processor is configured to obtain, from the stream of video data, a second distorted image depicting the object, and to provide the second distorted image to the adapted network to generate a second modified image. The object is distorted differently in the second distorted image than in the first distorted image. The processor is configured to generate a video output including the first modified image and the second modified image without visible artifacts due to distortion differences between the first distorted image and the second distorted image.

Abstract: A camera image processor receives frames from an image sensor of the camera. While continuing to receive frames from the image sensor, the camera image processor detecting an action within one or more frames of the received frames, for example using a convolution neural network trained to recognize one or more actions. While continuing to receive frames from the image sensor, the camera image processor captures the one or more frames with the detected action, for example as a still image, or as a slow-motion portion of a video that includes the received frames.

Abstract: Methods, systems, and devices for low-cost video segmentation are described. A media file may include multiple frames. Information for pixels in a first frame and pixels in a subsequent frame may be discarded based on segmentation maps computed for the first and subsequent frame. After discarding the pixel information, motion information may be determined for the remaining pixels in the first and subsequent frame. The segmentation maps generated for the first and subsequent frames and the determined motion information may be used to compute one or more additional segmentation maps for one or more additional frames that are temporally located between the first and subsequent frame. After computing segmentation maps for all or a portion of the frames in the media file, a modified version of the frames may be output based on the computed segmentation maps.

Abstract: A system comprising one or more storage devices configured to store data representing a video sequence, and one or more processors. The storage device(s) store instructions that, when executed, cause the at least one processor to: determine a region of interest for an object in a video frame of a video sequence, determine motion information between the video frame and a later video frame of the video sequence, determine, based on the region of interest and the motion information, an adjusted region of interest in the later video frame, and apply a mean shift algorithm to identify, based on the adjusted region of interest, the object in the later video frame.

Abstract: A method for generating a cinemagraph is described. The method includes determining optical flow information for a frame sequence. The method also includes performing image stabilization using the optical flow information to produce a stabilized frame sequence. The method further includes performing object tracking across the frame sequence using motion vector information to determine an object region of interest (ROI) of an object in each frame in the frame sequence. The method additionally includes generating a masking area for each frame in the stabilized frame sequence based on the object ROI for the frame. The method also includes merging the masking areas and the stabilized frame sequence to generate the cinemagraph.