Abstract: The present disclosure generally relates to image processing. For example, aspects of the present disclosure include systems and techniques for performing spatial and temporal processing of image data. Certain aspects provide an apparatus for processing frame data. The apparatus generally includes a memory, and one or more processors coupled to the memory, the one or more processors configured to: perform a first noise reduction operation based on first frame data via a machine learning component to generate first processed frame data; generate first feedback data based on the first processed frame data; and perform, via the machine learning component, a second noise reduction operation based on second frame data and the first feedback data.

Abstract: Temporal filtering operations may be reset for certain pixels within an image frame to reduce contribution from previous input frames to reduce ghosting and other artifacts. The resetting reduces the contribution to, for example, zero, either immediately or within a predetermined period of time (e.g., a certain number of frames). A decision regarding whether to reset temporal filtering for a pixel of the image frame may be based on a probability assigned to that pixel. The probability can be based on rules with one or more criteria. One example factor for adjusting probability is a confidence level regarding the temporal filtering decision for the pixel, in which the probability for a random reset of a pixel is based on the confidence level regarding the temporal filtering decision for those pixels.

Abstract: Temporal filtering operations may be reset for certain pixels within an image frame to reduce contribution from previous input frames to reduce ghosting and other artifacts. The resetting reduces the contribution to, for example, zero, either immediately or within a predetermined period of time (e.g., a certain number of frames). A decision regarding whether to reset temporal filtering for a pixel of the image frame may be based on a probability assigned to that pixel. The probability can be based on rules with one or more criteria. One example factor for adjusting probability is a confidence level regarding the temporal filtering decision for the pixel, in which the probability for a random reset of a pixel is based on the confidence level regarding the temporal filtering decision for those pixels.

Abstract: Techniques and systems are provided for processing one or more frames or images. For instance, a process for determining exposure for one or more frames includes obtaining a motion map for one or more frames. The process includes determining, based on the motion map, motion associated with the one or more frames of a scene. The motion corresponds to movement of one or more objects in the scene relative to a camera used to capture the one or more frames. The process includes determining, based on the determined motion, a number of frames and an exposure for capturing the number of frames. The process further includes sending a request to capture the number of frames using the determined exposure duration.

Abstract: In general, techniques are described regarding generating, based on zoom settings, various sets of frames of image data (e.g., video data). Cameras comprising one or more camera processors configured to perform the techniques are also disclosed. The camera processor(s) receive a plurality of frames of image data. The camera processor(s) receive, via a user interface, a zoom setting for each of the plurality of frames. The camera processor(s) may generate a first set of frames according to the zoom settings. The camera processor(s) determine a second set of frames, where the second set of frames are different from the first set of frames. The second set of frames may comprise a second zoom that is different than a first zoom applied to the first set of frames.

Abstract: In general, techniques are described regarding selective allocation of processing resources for processing image data. Processing circuitry may warp a frame of image data around variously indicated areas of the frame to create a warped frame of image data. The processing circuitry may allocate more pixels to the indicated areas by virtue of the frame warping operation. The warped frame may then be output for further processing in accordance with proportions of the warped frame resulting in better image or video quality in the indicated areas.

Abstract: In general, techniques are described regarding generating, based on zoom settings, various sets of frames of image data (e.g., video data). Cameras comprising one or more camera processors configured to perform the techniques are also disclosed. The camera processor(s) receive a plurality of frames of image data. The camera processor(s) receive, via a user interface, a zoom setting for each of the plurality of frames. The camera processor(s) may generate a first set of frames according to the zoom settings. The camera processor(s) determine a second set of frames, where the second set of frames are different from the first set of frames. The second set of frames may comprise a second zoom that is different than a first zoom applied to the first set of frames.

Abstract: In general, techniques are described regarding selective allocation of processing resources for processing image data. Processing circuitry may warp a frame of image data around variously indicated areas of the frame to create a warped frame of image data. The processing circuitry may allocate more pixels to the indicated areas by virtue of the frame warping operation. The warped frame may then be output for further processing in accordance with proportions of the warped frame resulting in better image or video quality in the indicated areas.

Abstract: A video encoder with an integrated temporal filter, which has an output carrying a sequence of encoded frames, includes: an input receiving input frame data; a temporal filter receiving the input frame data from the input and producing filtered frame data; motion processing modules receiving reconstructed frames derived from the sequence of encoded frames and the filtered frame data from the temporal filter and producing a residual error signal output; and a coder module which receives the input frame data and the residual error signal and produces the sequence of encoded frames.

Abstract: A system, method, and computer program product for digital stabilization of video data from cameras producing multiple simultaneous views, typically from rolling shutter type sensors, and without requiring a motion sensor. A first embodiment performs an estimation of the global transformation on a single view and uses this transformation for correcting other views. A second embodiment selects a distance at which a maximal number of scene points is located and considers only the motion vectors from these image areas for the global transformation. The global transformation estimate is improved by averaging images from several views and reducing stabilization when image conditions may cause incorrect stabilization. Intentional motion is identified confidently in multiple views. Local object distortion may be corrected using depth information. A third embodiment analyzes the depth of the scene and uses the depth information to perform stabilization for each of multiple depth layers separately.

Abstract: A method and device are provided for method for stabilization of image data by an imaging device. In one embodiment, a method includes detecting image data for a first frame and a second frame, performing motion estimation to determine one or more motion vectors associated with global frame motion for image data of the first frame, performing an outlier rejection function to select at least one of the one or more motion vectors, and determining a global transformation for image data of the first frame based, at least in part, on motion vectors selected by the outlier rejection function. The method may further include determining a stabilization transformation for image data of the first frame by refining the global transformation to correct for unintentional motion and applying the stabilization transformation to image data of the first frame to stabilize the image data of the first frame.

Abstract: A method and device are provided for method for stabilization of image data by an imaging device. In one embodiment, a method includes detecting image data for a first frame and a second frame, performing motion estimation to determine one or more motion vectors associated with global frame motion for image data of the first frame, performing an outlier rejection function to select at least one of the one or more motion vectors, and determining a global transformation for image data of the first frame based, at least in part, on motion vectors selected by the outlier rejection function. The method may further include determining a stabilization transformation for image data of the first frame by refining the global transformation to correct for unintentional motion and applying the stabilization transformation to image data of the first frame to stabilize the image data of the first frame.

Abstract: A video encoder with an integrated temporal filter, which has an output carrying a sequence of encoded frames, includes: an input receiving input frame data; a temporal filter receiving the input frame data from the input and producing filtered frame data; motion processing modules receiving reconstructed frames derived from the sequence of encoded frames and the filtered frame data from the temporal filter and producing a residual error signal output; and a coder module which receives the input frame data and the residual error signal and produces the sequence of encoded frames.