Abstract: Apparatus and methods related to image processing are provided. A computing device can determine a first image area of an image, such as an image captured by a camera. The computing device can determine a warping mesh for the image with a first portion of the warping mesh associated with the first image area. The computing device can determine a cost function for the warping mesh by: determining first costs associated with the first portion of the warping mesh that include costs associated with face-related transformations of the first image area to correct geometric distortions. The computing device can determine an optimized mesh based on optimizing the cost function. The computing device can modify the first image area based on the optimized mesh.

Abstract: Methods, systems, and apparatus, including computer programs stored on a computer-readable storage medium, for video stabilization. In some implementations, a computer system obtains frames of a video captured by a recording device using an optical image stabilization (OIS) system. The computing system receives (i) OIS position data indicating positions of the OIS system during capture of the frames, and (ii) device position data indicating positions of the recording device during capture of the frames. The computing system determines a first transformation for a particular frame based on the OIS position data for the particular frame and device position data for the particular frame. The computing system determines a second transformation for the particular frame based on the first transformation and positions of the recording device occurring after capture of the particular frame. The computing system generates a stabilized version of the particular frame using the second transformation.

Abstract: Implementations generally relate to locking a camera onto a sub-region to capture images of the sub-region with a moving camera. In some implementations, a method includes determining a first region in a scene detected by a camera while the camera is in a first physical position. The method further includes receiving user input to target a sub-region within the first capture region of the scene and capturing the first image of the sub-region while the camera is in the first physical position. The first image is displayed in a fixed orientation on a preview portion of a display screen. The camera is moved to a second physical position and a second capture region of the scene is detected. The method further includes capturing a second image is of the sub-region and displaying it in the fixed orientation on the preview portion of the display screen. The second image is effective to inform a user of whether capture of the sub-region is altered by camera movement.

Abstract: Methods, systems, and apparatus, including computer programs stored on a computer-readable storage medium, for video stabilization. In some implementations, a computer system obtains frames of a video captured by a recording device using an optical image stabilization (OIS) system. The computing system receives (i) OIS position data indicating positions of the OIS system during capture of the frames, and (ii) device position data indicating positions of the recording device during capture of the frames. The computing system determines a first transformation for a particular frame based on the OIS position data for the particular frame and device position data for the particular frame. The computing system determines a second transformation for the particular frame based on the first transformation and positions of the recording device occurring after capture of the particular frame. The computing system generates a stabilized version of the particular frame using the second transformation.

Abstract: Apparatus and methods related to image processing are provided. A computing device can determine a first image area of an image, such as an image captured by a camera. The computing device can determine a warping mesh for the image with a first portion of the warping mesh associated with the first image area. The computing device can determine a cost function for the warping mesh by: determining first costs associated with the first portion of the warping mesh that include costs associated with face-related transformations of the first image area to correct geometric distortions, and determining second costs associated with the warping mesh that include costs of edge-related transformations for preserving straightness of edges of the image. The computing device can determine an optimized mesh based on optimizing the cost function. The computing device can modify the first image area based on the optimized mesh.

Abstract: Apparatus and methods related to image processing are provided. A computing device can determine a first image area of an image, such as an image captured by a camera. The computing device can determine a warping mesh for the image with a first portion of the warping mesh associated with the first image area. The computing device can determine a cost function for the warping mesh by: determining first costs associated with the first portion of the warping mesh that include costs associated with face-related transformations of the first image area to correct geometric distortions. The computing device can determine an optimized mesh based on optimizing the cost function. The computing device can modify the first image area based on the optimized mesh.

Abstract: A light-field camera may generate four-dimensional light-field data indicative of incoming light. The light-field camera may have an aperture configured to receive the incoming light, an image sensor, and a microlens array configured to redirect the incoming light at the image sensor. The image sensor may receive the incoming light and, based on the incoming light, generate the four-dimensional light-field data, which may have first and second spatial dimensions and first and second angular dimensions. The first angular dimension may have a first resolution higher than a second resolution of the second angular dimension.

Abstract: The subject matter described in this disclosure can be embodied in methods and systems for stabilizing video. A computing system determines a stabilized location of a facial feature in a frame of video accounting for its location in a previous frame. The computing system determines a physical camera pose in virtual space and maps the frame into virtual space. The computing system determines an optimized virtual camera pose using an optimization process that determines (1) a difference between the stabilized location of the facial feature and a location of the facial feature when viewed from a potential virtual camera pose, (2) a difference between the potential virtual camera pose and a previous virtual camera pose, and (3) a difference between the potential virtual camera pose and the physical camera pose. The computing system generates the stabilized view of the frame using the optimized virtual camera pose.

Abstract: The subject matter described in this disclosure can be embodied in methods and systems for stabilizing video. A computing system determines a stabilized location of a facial feature in a frame of video accounting for its location in a previous frame. The computing system determines a physical camera pose in virtual space and maps the frame into virtual space. The computing system determines an optimized virtual camera pose using an optimization process that determines (1) a difference between the stabilized location of the facial feature and a location of the facial feature when viewed from a potential virtual camera pose, (2) a difference between the potential virtual camera pose and a previous virtual camera pose, and (3) a difference between the potential virtual camera pose and the physical camera pose. The computing system generates the stabilized view of the frame using the optimized virtual camera pose.

Abstract: Implementations generally relate to locking a camera onto a sub-region to capture images of the sub-region with a moving camera. In some implementations, a method includes determining a first region in a scene detected by a camera while the camera is in a first physical position. The method further includes receiving user input to target a sub-region within the first capture region of the scene and capturing the first image of the sub-region while the camera is in the first physical position. The first image is displayed in a fixed orientation on a preview portion of a display screen. The camera is moved to a second physical position and a second capture region of the scene is detected. The method further includes capturing a second image is of the sub-region and displaying it in the fixed orientation on the preview portion of the display screen. The second image is effective to inform a user of whether capture of the sub-region is altered by camera movement.

Abstract: An image such as a light-field image may be processed to provide depth-based blurring. The image may be received in a data store. At an input device, first and second user input may be received to designate a first focus depth and a second focus depth different from the first focus depth, respectively. A processor may identify one or more foreground portions of the image that have one or more foreground portion depths, each of which is less than the first focus depth. The processor may also identify one or more background portions of the image that have one or more background portion depths, each of which is greater than the second focus depth. The processor may also apply blurring to the one or more foreground portions and the one or more background portions to generate a processed image, which may be displayed on a display device.

Abstract: Implementations generally relate to locking a camera onto a sub-region to capture images of the sub-region with a moving camera. In some implementations, a method includes determining a first region in a scene detected by a camera while the camera is in a first physical position. The method further includes receiving user input to target a sub-region within the first capture region of the scene and capturing the first image of the sub-region while the camera is in the first physical position. The first image is displayed in a fixed orientation on a preview portion of a display screen. The camera is moved to a second physical position and a second capture region of the scene is detected. The method further includes capturing a second image is of the sub-region and displaying it in the fixed orientation on the preview portion of the display screen. The second image is effective to inform a user of whether capture of the sub-region is altered by camera movement.

Abstract: According to various embodiments, a light-field image may be compressed and/or decompressed to facilitate storage, transmission, or other functions related to the light-field image. A light-field image may be captured by a light-field image capture device having an image sensor and a microlens array. The light-field image may be received in a data store. A processor may generate a first refocus image pool with a plurality of refocus images based on the light-field image. The processor may further use the first refocus image pool to compress the light-field image to generate a bitstream, smaller than the light-field image, which is representative of the light-field image. The processor or a different processor may also be used to generate a second refocus image pool with a second plurality of images based on the bitstream. The second refocus image pool may be used to decompress the bitstream to generate a reconstructed light-field image.

Abstract: Methods, systems, and apparatus, including computer programs stored on a computer-readable storage medium, for video stabilization. In some implementations, a computer system obtains frames of a video captured by a recording device using an optical image stabilization (OIS) system. The computing system receives (i) OIS position data indicating positions of the OIS system during capture of the frames, and (ii) device position data indicating positions of the recording device during capture of the frames. The computing system determines a first transformation for a particular frame based on the OIS position data for the particular frame and device position data for the particular frame. The computing system determines a second transformation for the particular frame based on the first transformation and positions of the recording device occurring after capture of the particular frame. The computing system generates a stabilized version of the particular frame using the second transformation.

Abstract: Implementations generally relate to locking a camera onto a sub-region to capture images of the sub-region with a moving camera. In some implementations, a method includes determining a first region in a scene detected by a camera while the camera is in a first physical position. The method further includes receiving user input to target a sub-region within the first capture region of the scene and capturing the first image of the sub-region while the camera is in the first physical position. The first image is displayed in a fixed orientation on a preview portion of a display screen. The camera is moved to a second physical position and a second capture region of the scene is detected. The method further includes capturing a second image is of the sub-region and displaying it in the fixed orientation on the preview portion of the display screen. The second image is effective to inform a user of whether capture of the sub-region is altered by camera movement.

Abstract: The subject matter described in this disclosure can be embodied in methods and systems for stabilizing video. A computing system determines a stabilized location of a facial feature in a frame of video accounting for its location in a previous frame. The computing system determines a physical camera pose in virtual space and maps the frame into virtual space. The computing system determines an optimized virtual camera pose using an optimization process that determines (1) a difference between the stabilized location of the facial feature and a location of the facial feature when viewed from a potential virtual camera pose, (2) a difference between the potential virtual camera pose and a previous virtual camera pose, and (3) a difference between the potential virtual camera pose and the physical camera pose. The computing system generates the stabilized view of the frame using the optimized virtual camera pose.

Abstract: Methods, systems, and apparatus, including computer programs stored on a computer-readable storage medium, for video stabilization. In some implementations, a computer system obtains frames of a video captured by a recording device using an optical image stabilization (OIS) system. The computing system receives (i) OIS position data indicating positions of the OIS system during capture of the frames, and (ii) device position data indicating positions of the recording device during capture of the frames. The computing system determines a first transformation for a particular frame based on the OIS position data for the particular frame and device position data for the particular frame. The computing system determines a second transformation for the particular frame based on the first transformation and positions of the recording device occurring after capture of the particular frame. The computing system generates a stabilized version of the particular frame using the second transformation.

Abstract: A light-field camera may generate four-dimensional light-field data indicative of incoming light. The light-field camera may have an aperture configured to receive the incoming light, an image sensor, and a microlens array configured to redirect the incoming light at the image sensor. The image sensor may receive the incoming light and, based on the incoming light, generate the four-dimensional light-field data, which may have first and second spatial dimensions and first and second angular dimensions. The first angular dimension may have a first resolution higher than a second resolution of the second angular dimension.

Abstract: Methods, systems, and apparatus, including computer programs stored on a computer-readable storage medium, for video stabilization. In some implementations, a computer system obtains frames of a video captured by a recording device using an optical image stabilization (OIS) system. The computing system receives (i) OIS position data indicating positions of the OIS system during capture of the frames, and (ii) device position data indicating positions of the recording device during capture of the frames. The computing system determines a first transformation for a particular frame based on the OIS position data for the particular frame and device position data for the particular frame. The computing system determines a second transformation for the particular frame based on the first transformation and positions of the recording device occurring after capture of the particular frame. The computing system generates a stabilized version of the particular frame using the second transformation.

Abstract: According to various embodiments, the system and method disclosed herein process light-field image data so as to mitigate lens flare effects. A light-field image may be captured with a light-field image capture device with a microlens array and received in a data store. A plurality of flare-affected pixels may be identified in the light-field image. The flare-affected pixels may have flare-affected pixel values. Flare-corrected pixel values may be generated for the flare-affected pixels. Relative to the flare-affected pixel values, the flare-corrected pixel values may at least partially remove the lens flare effects. The flare-corrected pixel values may be used to generate a corrected light-field image in which the lens flare effects are at least partially corrected. The corrected light-field image may be displayed on a display screen.