Abstract: Light-field image data is processed in a manner that reduces projection artifacts in the presence of variation in microlens position by calibrating microlens positions. Initially, approximate centers of disks in a light-field image are identified. Gridded calibration is then performed, by fitting lines to disk centers along orthogonal directions, and then fitting a rigid grid to the light-field image. For each grid region, a corresponding disk center is computed by passing values for pixels within that grid region into weighted-center equations. A displacement vector is then generated, based on the distance from the geometric center of the grid region to the computed disk center. For each grid region, the final disk center is computed as the vector sum of the grid region's geometric center and displacement vector. Calibration data, including displacement vectors, is then used in calibrating disk centers for more accurate projection of light-field images.

Abstract: A video refocusing system operates in connection with refocusable video data, information, images and/or frames, which may be light field video data, information, images and/or frames, that may be focused and/or refocused after acquisition or recording. A video acquisition device acquires first refocusable light field video data of a scene, stores first refocusable video data representative of the first refocusable light field video data, acquires second refocusable light field video data of the scene after acquiring the first refocusable light field video data, determines a first virtual focus parameter (such as a virtual focus depth) using the second refocusable light field video data, generates first video data using the stored first refocusable video data and the first virtual focus parameter, wherein the first video data includes a focus depth that is different from an optical focus depth of the first refocusable light field video data, and outputs the first video data.

Abstract: According to various embodiments, the system and method of the present invention process light-field image data in a manner that reduces artifacts and that yields 2-D images with extended depth of field, and with variable placement of the center of perspective. Center of perspective can be varied based on user input or on pre-specified parameters. Various techniques for improving the presentation of light-field images with variable center of perspective are described, and for performing other effects in connection with projection of light-field images.

Abstract: A light field data acquisition device includes optics and a light field sensor to acquire light field image data of a scene. In at least one embodiment, the light field sensor is located at a substantially fixed, predetermined distance relative to the focal point of the optics. In response to user input, the light field acquires the light field image data of the scene, and a storage device stores the acquired data. Such acquired data can subsequently be used to generate a plurality of images of the scene using different virtual focus depths.

Abstract: According to various embodiments of the invention, improved downsampling techniques are employed, which can be applied to light field images and which preserve the ability to refocus (and otherwise manipulate) such images. Groups of pixels, rather than individual pixels, are downsampled; such groups of pixels can be defined, for example, as disks of pixels. Such downsampling is accomplished, for example, by aggregating values for pixels having similar relative positions within adjacent disks (or other defined regions or pixel groups) of the image. When applied to light field images, the downsampling techniques of the present invention reduce spatial resolution without sacrificing angular resolution. This ensures that the refocusing capability of the resulting light field image is not reduced and/or adversely impacted.

Abstract: According to various embodiments, the present may be used to apply a wide variety of processes to a two-dimensional image generated from light-field data. One or more parameters, such as light-field parameters and/or device capture parameters may be included in metadata of the two-dimensional image, and may be retrieved and processed to determine the appropriate value(s) of a first setting of the process. The process may be applied uniformly, or with variation across subsets of the two-dimensional image, down to individual pixels. The process may be a noise filtering process, an image sharpening process, a color adjustment process, a tone curve process, a contrast adjustment process, a saturation adjustment process, a gamma adjustment process, a combination thereof, or any other known process that may be desirable for enhancing two-dimensional images.

Abstract: The present invention operates in connection with refocusable video data, information, images and/or frames, which may be light field video data, information, images and/or frames, that may be focused and/or refocused after acquisition or recording. A video acquisition device acquires first refocusable light field video data of a scene, stores first refocusable video data representative of the first refocusable light field video data, acquires second refocusable light field video data of the scene after acquiring the first refocusable light field video data, determines a first virtual focus parameter (such as a virtual focus depth) using the second refocusable light field video data, generates first video data using the stored first refocusable video data and the first virtual focus parameter, wherein the first video data includes a focus depth that is different from an optical focus depth of the first refocusable light field video data, and outputs the first video data.

Abstract: A 3D stereo image of a scene is generated by generating first image data of the scene using light field data representing light rays from a first direction, and second image data of the scene using light field data representing light rays from a second direction. The 3D stereo image of the scene is then generated using the first image data and the second image data. A microlens array may be used to direct light rays onto a photosensor array, wherein each microlens of the microlens array includes a physical aperture which correlates to a plurality of associated photosensors, a first virtual aperture which correlates to a first subset of the associated photosensors, and a second virtual aperture which correlates to a second subset of the associated photosensors. Each virtual aperture thus provides light rays from a different direction for use in generating the 3D stereo image.

Abstract: A light field data acquisition device includes optics and a light field sensor to acquire light field image data of a scene. In at least one embodiment, the light field sensor is located at a substantially fixed, predetermined distance relative to the focal point of the optics. In response to user input, the light field acquires the light field image data of the scene, and a storage device stores the acquired data. Such acquired data can subsequently be used to generate a plurality of images of the scene using different virtual focus depths.

Abstract: The present invention operates in connection with refocusable video data, information, images and/or frames, which may be light field video data, information, images and/or frames, that may be focused and/or refocused after acquisition or recording. A video acquisition device acquires first refocusable light field video data of a scene, stores first refocusable video data representative of the first refocusable light field video data, acquires second refocusable light field video data of the scene after acquiring the first refocusable light field video data, determines a first virtual focus parameter (such as a virtual focus depth) using the second refocusable light field video data, generates first video data using the stored first refocusable video data and the first virtual focus parameter, wherein the first video data includes a focus depth that is different from an optical focus depth of the first refocusable light field video data, and outputs the first video data.

Abstract: Systems and methods are directed to acquiring, generating, manipulating and/or editing video data/frames. The video frames may be light field video frames having virtual aperture parameters that may be adjusted after acquisition or recording of such video frames. In one aspect, a method comprises: selecting a first key frame, wherein the first key frame corresponds to one of a plurality of light field video frames; selecting a second key frame, wherein the second key frame corresponds to one of the plurality of light field video frames which is temporally spaced apart from the first key frame such that a plurality of light field video frames are temporally disposed between the first and the second key frames; determining virtual aperture parameters for the first key frame and the second key frame; and generating video data.

Abstract: Certain systems and methods are directed to acquiring, generating, manipulating and/or editing (for example, focusing or refocusing) refocusable video data, information, images and/or frames. The refocusable video data, information, images and/or frames may be light field video data, information, images and/or frames, that may be focused and/or re-focused after acquisition or recording of such video data, information, images and/or frames.

Abstract: Systems and methods are directed to acquiring, generating, manipulating and/or editing video data/frames. The video frames may be light field video frames having virtual aperture parameters that may be adjusted after acquisition or recording of such video frames. In one aspect, a method comprises: selecting a first key frame, wherein the first key frame corresponds to one of a plurality of light field video frames; selecting a second key frame, wherein the second key frame corresponds to one of the plurality of light field video frames which is temporally spaced apart from the first key frame such that a plurality of light field video frames are temporally disposed between the first and the second key frames; determining virtual aperture parameters for the first key frame and the second key frame; and generating video data.

Abstract: Systems and methods are directed to acquiring, generating, manipulating and/or editing refocusable video data/frames. The refocusable video frames may be light field video frames that may be focused and/or refocused after acquisition or recording of such video frames. In one aspect, a method comprises: selecting a first key frame, wherein the first key frame corresponds to one of a plurality of refocusable light field video frames; selecting a second key frame, wherein the second key frame corresponds to one of the plurality of refocusable light field video frames which is temporally spaced apart from the first key frame such that a plurality of refocusable light field video frames are temporally disposed between the first and the second key frames; determining virtual focus parameters for the first key frame and the second key frame; and generating video data.

Abstract: An optical assembly includes a solid spacing layer between a plenoptic microlens array (MLA) and a pixel-level MLA, avoiding the need for an air gap. Such an assembly, and systems and methods for manufacturing same, can yield improved reliability and efficiency of production, and can avoid many of the problems associated with prior art approaches. In at least one embodiment, the plenoptic MLA, the spacing layer, and the pixel-level MLA are created from optically transmissive polymer(s) deposited on the photosensor array and shaped using photolithographic techniques. Such an approach improves precision in placement and dimensions, and avoids other problems associated with conventional polymer-on-glass architectures. Further variations and techniques are described.

Abstract: A light field data acquisition device includes optics and a light field sensor to acquire light field image data of a scene. In at least one embodiment, the light field sensor is located at a substantially fixed, predetermined distance relative to the focal point of the optics. In response to user input, the light field acquires the light field image data of the scene, and a storage device stores the acquired data. Such acquired data can subsequently be used to generate a plurality of images of the scene using different virtual focus depths.

Abstract: Systems and methods are directed to acquiring, generating, manipulating and/or editing refocusable video data/frames. The refocusable video frames may be light field video frames that may be focused and/or refocused after acquisition or recording of such video frames. In one aspect, a method comprises: selecting a first key frame, wherein the first key frame corresponds to one of a plurality of refocusable light field video frames; selecting a second key frame, wherein the second key frame corresponds to one of the plurality of refocusable light field video frames which is temporally spaced apart from the first key frame such that a plurality of refocusable light field video frames are temporally disposed between the first and the second key frames; determining virtual focus parameters for the first key frame and the second key frame; and generating video data.

Abstract: A light field data acquisition device includes optics and a sensor to acquire light field image data of a scene. In at least one embodiment, the sensor is located at a substantially fixed, predetermined location relative to the focal point of the optics. The device determines a first virtual focus depth different from an optical focus depth of the light field image data, and automatically generates and outputs data representative of a first image of the scene at the first virtual focus depth. In response to user input, the device determines a second virtual focus depth and generates data representative of a second image of the scene at the second virtual focus depth.

Abstract: A system and method are provided for storing, manipulating, and/or transmitting image data, such as light field photographs and the like, in a manner that efficiently delivers different capabilities and features based on device attributes, user requirements and preferences, context, and/or other factors. Acceleration structures are provided, which enable selective use of certain types of data (also referred to as “assets”) based on device attributes such as image size, desired functionality, user preference, and/or the like. In this manner, the system and method of the present invention takes into account specific attributes and parameters in determining which data should be included, so as to optimize transmission, storage, and/or rendering of image data, including light field data, to improve efficiency and avoid waste of resources.

Abstract: Systems and methods directed to acquiring, generating, manipulating and/or editing refocusable video data/frames. The refocusable video frames may be light field video frames that may be focused and/or re-focused after acquisition or recording of such video frames. In one aspect, method comprises: selecting a first key frame, wherein the first key frame corresponds to one of a plurality of refocusable light field video frames, selecting a second key frame, wherein the second key frame corresponds to one of the plurality of refocusable light field video frames which is temporally spaced apart from the first key frame such that a plurality of refocusable light field video frames are temporally disposed between the first and the second key frames, determining a virtual focus parameters for the first key frame and the second key frame, and generating first video data.