Abstract: Video information defining video content to be encoded may be obtained. Scene composition information for the video content may be obtained. The scene composition information may be determined by a convolutional neural network based on visuals represented within the video content. The video content may be encoded based on the scene composition information. The encoding of the video content may generate encoded video information defining the encoded video content.

Abstract: Systems and methods are disclosed for image signal processing. For example, methods may include receiving, by an image signal processor, one or more input image signals from one or more image sensors; determining a mapping based on the input image signal(s), wherein the mapping includes records that associate image portions of an output image with corresponding image portions of the input image signal(s); sorting the records of the mapping according to an order of the corresponding image portions of the input image signal(s); applying, by the image signal processor, image processing to image portions of the input image signal(s) to determine image portions of one or more processed images in the order; and determining, by the image signal processor, the image portions of the output image based at least in part on the mapping and the corresponding image portions of the processed image(s) in the order.

Abstract: Image signal processing including generating image signal processing based encoding hints for motion estimation may include an image signal processor obtaining an input image portion of an input image from the input image signal, generating motion information for the input image portion, processing the input image portion based on the motion information, outputting processed image data, and outputting the motion information as encoding hints, such that the motion information is accessible by an encoder for generating an encoded output bitstream by obtaining the processed image data as source image data, obtaining the motion information, generating prediction data for encoding the source image data based on the motion information, generating encoded image data based on the prediction data, and including the encoded image data in an encoded output bitstream.

Abstract: Video information defining video content to be encoded may be obtained. Scene composition information for the video content may be obtained. The scene composition information may be determined by a convolutional neural network based on visuals represented within the video content. The video content may be encoded based on the scene composition information. The encoding of the video content may generate encoded video information defining the encoded video content.

Abstract: Systems and methods are disclosed for image signal processing. For example, methods may include receiving, by an image signal processor, one or more input image signals from one or more image sensors; determining a mapping based on the input image signal(s), wherein the mapping includes records that associate image portions of an output image with corresponding image portions of the input image signal(s); sorting the records of the mapping according to an order of the corresponding image portions of the input image signal(s); applying, by the image signal processor, image processing to image portions of the input image signal(s) to determine image portions of one or more processed images in the order; and determining, by the image signal processor, the image portions of the output image based at least in part on the mapping and the corresponding image portions of the processed image(s) in the order.

Abstract: Video information defining video content to be encoded may be obtained. Scene composition information for the video content may be obtained. The scene composition information may be determined by a convolutional neural network based on visuals represented within the video content. The video content may be encoded based on the scene composition information. The encoding of the video content may generate encoded video information defining the encoded video content.

Abstract: Systems and methods for utilizing on-camera sensor information to improve video and/or image encoding quality are discussed herein. Specifically, the systems and methods may utilize on-camera sensor information to efficiently determine whether to encode a particular frame within a set of frames as an intra frame. When captured on video, a particular arrangement of a group of pixels within a frame may comprise a visual representation of an object within the frame. When encoding video footage, motion information characterizing motion of an image capturing device over time may be used to predict displacement of an arrangement of group of pixels between frames. These predictions may be used to efficiently determine whether to encode a particular frame as an intra frame.

Abstract: A camera system processes images based on image luminance data. The camera system includes an image sensor, an image pipeline, an encoder and a memory. The image sensor converts light incident upon the image sensor into raw image data. The image pipeline converts raw image data into color-space image data and calculates luminance levels of the color-space image data. The encoder can determine one or more of quantization levels, determining GOP structure or reference frame spacing for the color-space image data based on the luminance levels. The memory stores the color-space image data and the luminance levels.

Abstract: Image signal processing based encoding hints for bitrate control may include an image signal processor receiving an input image signal, identifying a block of a frame based on the input image signal, generating processed image data corresponding to the block, determining a processed image indication for the block, and outputting the processed image data and the processed image indication as an encoding hint. Image signal processing based encoding hints for bitrate control may include an encoder receiving source image data for a frame, the source image data including the processed image data corresponding to the block, receiving source image indications for the frame, the source image indications including the processed image indication, determining an encoding parameter for encoding the frame based on the processed image indication, generating an encoded frame by encoding the frame using the encoding parameter, and outputting or storing the encoded frame.

Abstract: Image signal processing including generating image signal processing based encoding hints for motion estimation may include an image signal processor obtaining an input image portion of an input image from the input image signal, generating motion information for the input image portion, processing the input image portion based on the motion information, outputting processed image data, and outputting the motion information as encoding hints, such that the motion information is accessible by an encoder for generating an encoded output bitstream by obtaining the processed image data as source image data, obtaining the motion information, generating prediction data for encoding the source image data based on the motion information, generating encoded image data based on the prediction data, and including the encoded image data in an encoded output bitstream.

Abstract: Systems and methods for utilizing on-camera sensor information to improve video and/or image encoding quality are discussed herein. Specifically, the systems and methods described herein may utilize on-camera sensor information to adaptively adjust an intra frame insertion rate associated with a sequence of frames. The intra frame insertion rate associated with a sequence of frames may be adjusted based on the motion of the image capturing device while capturing the sequence of frames and a predefined motion threshold associated with the intra frame insertion rate of the sequence of frames. In some implementations, the intra frame insertion rate may be adjusted based on the activity being performed during the capture of the sequence of frames. As such, the encoding of one or more frames within a sequence of frames may be adaptively adjusted to better suit the scene depicted by the sequence of frames.

Abstract: Systems and methods are disclosed for image signal processing. For example, methods may include receiving, by an image signal processor, one or more input image signals from one or more image sensors; determining a mapping based on the input image signal(s), wherein the mapping includes records that associate image portions of an output image with corresponding image portions of the input image signal(s); sorting the records of the mapping according to an order of the corresponding image portions of the input image signal(s); applying, by the image signal processor, image processing to image portions of the input image signal(s) to determine image portions of one or more processed images in the order; and determining, by the image signal processor, the image portions of the output image based at least in part on the mapping and the corresponding image portions of the processed image(s) in the order.

Abstract: A camera system processes images based on image luminance data. The camera system includes an image sensor, an image pipeline, an encoder and a memory. The image sensor converts light incident upon the image sensor into raw image data. The image pipeline converts raw image data into color-space image data and calculates luminance levels of the color-space image data. The encoder can determine one or more of quantization levels, determining GOP structure or reference frame spacing for the color-space image data based on the luminance levels. The memory stores the color-space image data and the luminance levels.

Abstract: A camera system processes images based on image luminance data. The camera system includes an image sensor, an image pipeline, an encoder and a memory. The image sensor converts light incident upon the image sensor into raw image data. The image pipeline converts raw image data into color-space image data and calculates luminance levels of the color-space image data. The encoder can determine one or more of quantization levels, determining GOP structure or reference frame spacing for the color-space image data based on the luminance levels. The memory stores the color-space image data and the luminance levels.

Abstract: A camera system processes images based on image activity data. The camera system includes an image sensor, an image pipeline, an encoder and a memory. The image sensor converts light incident upon the image sensor into raw image data. The image pipeline converts raw image data into color-space image data and calculates activity variances of the color-space image data. The encoder can determine one or more of quantization levels, block type (Intra vs Inter), determining transform size and type, and determining GOP structure or reference frame spacing for the color-space image data based on the activity variances. The memory stores the color-space image data and the activity variances.

Abstract: Example methods and apparatuses including a picture parallel decoder are described herein. The picture parallel decoder includes an entropy pre-processing unit, an entropy transcoding unit, and a plurality of decoders. The entropy pre-processing unit may be configured to determine dependencies between frames of an encoded bitstream and to determine slice location data within the encoded bitstream based on dependencies between the frames. The entropy transcoding unit may be configured to transcode slices of the encoded bitstream based on the dependencies between frames. The plurality of decoders may be configured to decode two or more of the transcoded slices in parallel.

Abstract: Methods and apparatus for the encoding of imaging data using pre-stored imaging statistics. Many extant image capture devices, including without limitation, smartphones, handheld video cameras, and other types of image capture devices, typically include, for example, auto-exposure (AE), auto-white balance (AWB) and auto-focus (AF) modules in an image signal processing (ISP) pipeline. These modules within the ISP pipeline generate various imaging statistics which can be repurposed for the encoding process of video data. These imaging statistics can be utilized for a number of encoding processes including, without limitation, adjusting an encoder parameter value for the encoding process, adjustment of the motion estimation search range, insertion of intra-frames within the video data and the determination of whether to use explicit or implicit weighting prediction.

Abstract: Methods and apparatus for the stitching of images from a multi-camera array. In one embodiment, stitching is performed for a first image and a second image with an overlapping field of view by: encoding the first image to produce a first encoded image; encoding the second image to produce a second uncompressed encoded image; stitching the first image with the second image by: decoding the first encoded image to produce a decoded first image; storing the decoded first image in memory; accessing, by a stitching engine, the decoded first image from memory; accessing, by the stitching engine, the second uncompressed encoded image; stitching, by the stitching engine, the decoded first image with the accessed second uncompressed encoded image to produce a stitched image; and outputting, by the stitching engine, the stitched image.

Abstract: Systems and methods are disclosed for image signal processing. For example, methods may include receiving, by an image signal processor, one or more input image signals from one or more image sensors; determining a mapping based on the input image signal(s), wherein the mapping includes records that associate image portions of an output image with corresponding image portions of the input image signal(s); sorting the records of the mapping according to an order of the corresponding image portions of the input image signal(s); applying, by the image signal processor, image processing to image portions of the input image signal(s) to determine image portions of one or more processed images in the order; and determining, by the image signal processor, the image portions of the output image based at least in part on the mapping and the corresponding image portions of the processed image(s) in the order.

Abstract: A camera system processes images based on image activity data. The camera system includes an image sensor, an image pipeline, an encoder and a memory. The image sensor converts light incident upon the image sensor into raw image data. The image pipeline converts raw image data into color-space image data and calculates activity variances of the color-space image data. The encoder can determine one or more of quantization levels, block type (Intra vs Inter), determining transform size and type, and determining GOP structure or reference frame spacing for the color-space image data based on the activity variances. The memory stores the color-space image data and the activity variances.