Abstract: According to various embodiments, the system and method disclosed herein process light-field image data so as to prevent, mitigate, and/or remove artifacts and/or other image degradation effects. A light-field image may be captured with a light-field image capture device with a microlens array. Based on a depth map of the light-field image, a plurality of layers may be created, and samples from the light-field image may be projected onto the layers to create a plurality of layer images. The layer images may be processed with one or more algorithms such as an inpainting algorithm to fill null values, a reconstruction algorithm to correct degradation effects from capture, and/or an enhancement algorithm to adjust the color, brightness, contrast, and/or sharpness of the layer image. Then, the layer images may be combined to generate a processed light-field image.

Abstract: Microlens positions for a light-field capture device may be calibrated. A calibration light-field image may be captured, with a microlens portion corresponding to each microlens of the light-field capture device. Interstitial spaces between the microlens portions may be identified and used to locate one or more center locations of the microlens portions. The center locations may be used to generate a model that indicates the microlens positions. Additionally or alternatively, the calibration light field image may be used to select one or more contour samples from among multiple contour samples of the microlens portions. The contour sample may be fitted to a circle centered at a center location of a microlens portion to identify the center location, which may then be used to generate a model that indicates the microlens positions. Multiple iterations may be used to enhance the accuracy of the models.

Abstract: RAW images and/or light field images may be compressed through the use of specialized techniques. The color depth of a light field image may be reduced through the use of a bit reduction algorithm such as a K-means algorithm. The image may then be retiled to group pixels of similar intensities and/or colors. The retiled image may be padded with extra pixel rows and/or pixel columns as needed, and compressed through the use of an image compression algorithm. The compressed image may be assembled with metadata pertinent to the manner in which compression was done to form a compressed image file. The compressed image file may be decompressed by following the compression method in reverse.

Abstract: According to various embodiments, a light-field image may be compressed and/or decompressed to facilitate storage, transmission, or other functions related to the light-field image. A light-field image may be captured by a light-field image capture device having an image sensor and a microlens array. The light-field image may be received in a data store. A processor may generate a first refocus image pool with a plurality of refocus images based on the light-field image. The processor may further use the first refocus image pool to compress the light-field image to generate a bitstream, smaller than the light-field image, which is representative of the light-field image. The processor or a different processor may also be used to generate a second refocus image pool with a second plurality of images based on the bitstream. The second refocus image pool may be used to decompress the bitstream to generate a reconstructed light-field image.

Abstract: According to various embodiments, the system and method disclosed herein facilitate the design of plenoptic camera lens systems to enhance camera resolution. A first configuration for the plenoptic camera may first be selected, with a first plurality of variables that define attributes of the plenoptic camera. The attributes may include a main lens attribute of a main lens of the plenoptic camera and/or a phase mask attribute of a phase mask of the plenoptic camera. A merit function may be applied by simulating receipt of light through the main lens and the plurality of microlenses of the first configuration to calculate a first merit function value. The main lens attribute and/or the phase mask attribute may be iteratively perturbed, and the merit function may be re-applied. An optimal set of variables may be identified by comparing results of successive applications of the merit function.

Abstract: According to various embodiments, the system and method of the present invention process light-field image data so as to reduce color artifacts, reduce projection artifacts, and/or increase dynamic range. These techniques operate, for example, on image data affected by sensor saturation and/or microlens modulation. Flat-field images are captured and converted to modulation images, and then applied on a per-pixel basis, according to techniques described herein.

Abstract: According to various embodiments, the system and method disclosed herein process light-field image data so as to prevent, mitigate, and/or remove artifacts and/or other image degradation effects. A light-field image may be captured with a light-field image capture device with a microlens array. Based on a depth map of the light-field image, a plurality of layers may be created, and samples from the light-field image may be projected onto the layers to create a plurality of layer images. The layer images may be processed with one or more algorithms such as an inpainting algorithm to fill null values, a reconstruction algorithm to correct degradation effects from capture, and/or an enhancement algorithm to adjust the color, brightness, contrast, and/or sharpness of the layer image. Then, the layer images may be combined to generate a processed light-field image.

Abstract: Microlens positions for a light-field capture device may be calibrated. A calibration light-field image may be captured, with a microlens portion corresponding to each microlens of the light-field capture device. Interstitial spaces between the microlens portions may be identified and used to locate one or more center locations of the microlens portions. The center locations may be used to generate a model that indicates the microlens positions. Additionally or alternatively, the calibration light field image may be used to select one or more contour samples from among multiple contour samples of the microlens portions. The contour sample may be fitted to a circle centered at a center location of a microlens portion to identify the center location, which may then be used to generate a model that indicates the microlens positions. Multiple iterations may be used to enhance the accuracy of the models.

Abstract: RAW images and/or light field images may be compressed through the use of specialized techniques. The color depth of a light field image may be reduced through the use of a bit reduction algorithm such as a K-means algorithm. The image may then be retiled to group pixels of similar intensities and/or colors. The retiled image may be padded with extra pixel rows and/or pixel columns as needed, and compressed through the use of an image compression algorithm. The compressed image may be assembled with metadata pertinent to the manner in which compression was done to form a compressed image file. The compressed image file may be decompressed by following the compression method in reverse.

Abstract: A system and method are provided for coordinating image capture using multiple devices, including for example multiple image capture devices (cameras), multiple lighting devices (flash), and/or the like. In at least one embodiment, the system of the present invention is configured to collect image information from multiple image capture devices, such as cameras, and/or to collect multiple images having different lighting configurations. The collected image data can be processed to generate various effects, such as relighting, parallax, refocusing, and/or three-dimensional effects, and/or to introduce interactivity into the image presentation. In at least one embodiment, the system of the present invention is implemented using any combination of any number of image capture device(s) and/or flash (lighting) device(s), which may be equipped to communicate with one another via any suitable means, such as wirelessly.

Abstract: Light-field image data is processed in a manner that reduces projection artifacts in the presence of variation in microlens position by calibrating microlens positions. Approximate centers of disks in a light-field image are identified, and gridded calibration is performed, by fitting lines to disk centers along orthogonal directions, and then fitting a rigid grid to the light-field image. For each grid region, a corresponding disk center is computed, and a displacement vector is generated. For each grid region, the final disk center is computed as the vector sum of the grid region's geometric center and displacement vector. Calibration data, including displacement vectors, is then used in calibrating disk centers for more accurate projection of light-field images. In at least one embodiment, the imaging geometry is arranged so that disks are separated by a gap, so as to limit or eliminate ghosting.

Abstract: According to various embodiments, the system and method disclosed herein serve to at least partially compensate for departures of an actual main lens of a light-field camera from the properties of an ideal main lens. Light-field data may be captured and processed through the use of product calibration data and unit calibration data. The product calibration data may be descriptive of departure of a main lens design of the light-field camera from an ideal main lens design. The unit calibration data may be descriptive of departure of the actual main lens of the light-field camera from the main lens design. Corrected light-field data may be generated as a result of the processing, and may be used to generate a light-field image.

Abstract: According to various embodiments, the system and method disclosed herein facilitate the design of plenoptic camera lens systems to enhance camera resolution. A first configuration for the plenoptic camera may first be selected, with a first plurality of variables that define attributes of the plenoptic camera. The attributes may include a main lens attribute of a main lens of the plenoptic camera and/or a phase mask attribute of a phase mask of the plenoptic camera. A merit function may be applied by simulating receipt of light through the main lens and the plurality of microlenses of the first configuration to calculate a first merit function value. The main lens attribute and/or the phase mask attribute may be iteratively perturbed, and the merit function may be re-applied. An optimal set of variables may be identified by comparing results of successive applications of the merit function.

Abstract: According to various embodiments, the system and method of the present invention process light-field image data so as to reduce color artifacts, reduce projection artifacts, and/or increase dynamic range. These techniques operate, for example, on image data affected by sensor saturation and/or microlens modulation. Flat-field images are captured and converted to modulation images, and then applied on a per-pixel basis, according to techniques described herein.

Abstract: A system and method are provided for coordinating image capture using multiple devices, including for example multiple image capture devices (cameras), multiple lighting devices (flash), and/or the like. In at least one embodiment, the system of the present invention is configured to collect image information from multiple image capture devices, such as cameras, and/or to collect multiple images having different lighting configurations. The collected image data can be processed to generate various effects, such as relighting, parallax, refocusing, and/or three-dimensional effects, and/or to introduce interactivity into the image presentation. In at least one embodiment, the system of the present invention is implemented using any combination of any number of image capture device(s) and/or flash (lighting) device(s), which may be equipped to communicate with one another via any suitable means, such as wirelessly.

Abstract: According to various embodiments of the invention, parallax and/or three-dimensional effects are added to thumbnail image displays. In at least one embodiment, such effects are applied in a manner that causes the thumbnail images to appear to respond to their display environment. For example, a parallax effect can be applied that responds to current cursor position, scroll position, scroll velocity, orientation of the display device (detected, for example, by position- and/or motion-sensing mechanisms), and/or any other environmental conditions. As another example, thumbnail images can be refocused, and/or a viewpoint for an image can be adjusted, in response to a user clicking on or tapping on particular elements within such images.

Abstract: Depth information is determined for elements in a light field image, thus allowing for rapid display of visualization tools to communicate such depth information to a user. Depth of strong edges within the light field image is analyzed, providing improved reliability of depth information while reducing or minimizing the amount of computation involved in generating such information. Strong edges can be identified and analyzed by generating epipolar images, or EPIs, from the light field image. Local gradients are determined for pixels in the EPIs. The magnitude of the local gradient is used to determine a confidence as to whether depth can be reliably estimated from the gradient. The orientation of the gradient is used to determine the depth of a corresponding element of the scene. Suitable output is then generated based on the determined depths, for example to provide information and feedback to aid a user in capturing light-field images.

Abstract: According to various embodiments, a dolly zoom effect is generated using light field image data. The dolly zoom effect simulates an in-camera technique wherein a camera moves toward or away from the subject in such a way that the subject is kept at the same size throughout the effect. The effect causes the relative size of foreground background elements to change while foreground elements such as the subject remain the same size. By varying a parameter while projecting the light field image, the size of each object in the projection image scales depending on its relative depth as compared with the depth of the target subject, thus simulating the dolly zoom effect without any need to physically move the camera.

Abstract: According to various embodiments, the system and method of the present invention process light-field image data so as to reduce color artifacts, reduce projection artifacts, and/or increase dynamic range. These techniques operate, for example, on image data affected by sensor saturation and/or microlens modulation. Flat-field images are captured and converted to modulation images, and then applied on a per-pixel basis, according to techniques described herein.

Abstract: Light-field image data is processed in a manner that reduces projection artifacts in the presence of variation in microlens position by calibrating microlens positions. Approximate centers of disks in a light-field image are identified, and gridded calibration is performed, by fitting lines to disk centers along orthogonal directions, and then fitting a rigid grid to the light-field image. For each grid region, a corresponding disk center is computed, and a displacement vector is generated. For each grid region, the final disk center is computed as the vector sum of the grid region's geometric center and displacement vector. Calibration data, including displacement vectors, is then used in calibrating disk centers for more accurate projection of light-field images. In at least one embodiment, the imaging geometry is arranged so that disks are separated by a gap, so as to limit or eliminate ghosting.