Abstract: Described herein are methods and devices that employ a plurality of image sensors to capture a target image of a scene. As described, positioning at least one reflective or refractive surface near the plurality of image sensors enables the sensors to capture together an image of wider field of view and longer focal length than any sensor could capture individually by using the reflective or refractive surface to guide a portion of the image scene to each sensor. The different portions of the scene captured by the sensors may overlap, and may be aligned and cropped to generate the target image.

Abstract: Methods and apparatus for reducing plenoptic camera artifacts. A first method is based on careful design of the optical system of the focused plenoptic camera to reduce artifacts that result in differences in depth in the microimages. A second method is computational; a focused plenoptic camera rendering algorithm is provided that corrects for artifacts resulting from differences in depth in the microimages. While both the artifact-reducing focused plenoptic camera design and the artifact-reducing rendering algorithm work by themselves to reduce artifacts, the two approaches may be combined.

Abstract: Methods and apparatus for capturing and rendering high-quality photographs using relatively small, thin plenoptic cameras. Plenoptic camera technology, in particular focused plenoptic camera technology including but not limited to super-resolution techniques, and other technologies such as microsphere technology may be leveraged to provide thin form factor, megapixel resolution cameras suitable for use in mobile devices and other applications. In addition, at least some embodiments of these cameras may also capture radiance, allowing the imaging capabilities provided by plenoptic camera technology to be realized through appropriate rendering techniques.

Abstract: Methods and apparatus for capturing and rendering high-quality photographs using relatively small, thin plenoptic cameras. Plenoptic camera technology, in particular focused plenoptic camera technology including but not limited to super-resolution techniques, and other technologies such as solid immersion lens (SIL) technology may be leveraged to provide thin form factor, megapixel resolution cameras suitable for use in mobile devices and other applications. In addition, at least some embodiments of these cameras may also capture radiance, allowing the imaging capabilities provided by plenoptic camera technology to be realized through appropriate rendering techniques. Hemispherical SIL technology, along with multiple main lenses and a mask on the photosensor, may be employed in some thin plenoptic cameras. Other thin cameras may include a layer between hemispherical SILs and the photosensor that effectively implements superhemispherical SIL technology in the camera.

Abstract: Methods, apparatus, and computer-readable storage media for rendering focused plenoptic camera data. A rendering with blending technique is described that blends values from positions in multiple microimages and assigns the blended value to a given point in the output image. A rendering technique that combines depth-based rendering and rendering with blending is also described. Depth-based rendering estimates depth at each microimage and then applies that depth to determine a position in the input flat from which to read a value to be assigned to a given point in the output image. The techniques may be implemented according to parallel processing technology that renders multiple points of the output image in parallel. In at least some embodiments, the parallel processing technology is graphical processing unit (GPU) technology.

Abstract: Methods, apparatus, and computer-readable storage media for rendering focused plenoptic camera data. A depth-based rendering technique is described that estimates depth at each microimage and then applies that depth to determine a position in the input flat from which to read a value to be assigned to a given point in the output image. The techniques may be implemented according to parallel processing technology that renders multiple points of the output image in parallel. In at least some embodiments, the parallel processing technology is graphical processing unit (GPU) technology.

Abstract: Methods, apparatus, and computer-readable storage media for calibrating focused plenoptic camera data. A calibration technique that does not modify the image data may be applied to raw plenoptic images. Calibration parameters, including but not limited to tilt angle, corner crops, main lens distance from the microlens array, sensor distance from the microlens array, and microimage size, may be specified. Calibration may include scaling down the input texture coordinates for the plenoptic image so that the new coordinate range fits the size of the texture with crops taken into account. These coordinates may be further transformed by one or more of a matrix performing a scaling, to correct for lens distortion; a rotation, to correct for tilts; and a translation that finalizes the necessary corner crops. A transformation matrix is generated that can be applied to the raw image by radiance processing techniques such as super-resolution techniques.

Abstract: A super-resolved demosaicing technique for rendering focused plenoptic camera data performs simultaneous super-resolution and demosaicing. The technique renders a high-resolution output image from a plurality of separate microimages in an input image at a specified depth of focus. For each point on an image plane of the output image, the technique determines a line of projection through the microimages in optical phase space according to the current point and angle of projection determined from the depth of focus. For each microimage, the technique applies a kernel centered at a position on the current microimage intersected by the line of projection to accumulate, from pixels at each microimage covered by the kernel at the respective position, values for each color channel weighted according to the kernel. A value for a pixel at the current point in the output image is computed from the accumulated values for the color channels.

Abstract: Methods and apparatus for super-resolution in integral photography are described. Several techniques are described that, alone or in combination, may improve the super-resolution process and/or the quality of super-resolved images that may be generated from flats captured with a focused plenoptic camera using a super-resolution algorithm. At least some of these techniques involve modifications to the focused plenoptic camera design. In addition, at least some of these techniques involve modifications to the super-resolution rendering algorithm. The techniques may include techniques for reducing the size of pixels, techniques for shifting pixels relative to each other so that super-resolution is achievable at more or all depths of focus, and techniques for sampling using an appropriate filter or kernel. These techniques may, for example, reduce or eliminate the need to perform deconvolution on a super-resolved image, and may improve super-resolution results and/or increase performance.

Abstract: Described herein are methods and devices that employ a plurality of image sensors to capture a target image of a scene. As described, positioning at least one reflective or refractive surface near the plurality of image sensors enables the sensors to capture together an image of wider field of view and longer focal length than any sensor could capture individually by using the reflective or refractive surface to guide a portion of the image scene to each sensor. The different portions of the scene captured by the sensors may overlap, and may be aligned and cropped to generate the target image.

Abstract: Methods, apparatus, and computer-readable storage media for simulating artistic effects in images rendered from plenoptic data. An impressionistic-style artistic effect may be generated in output images of a rendering process by an “impressionist” 4D filter applied to the microimages in a flat captured with focused plenoptic camera technology. Individual pixels are randomly selected from blocks of pixels in the microimages, and only the randomly selected pixels are used to render an output image. The randomly selected pixels are rendered to generate the artistic effect, such as an “impressionistic” effect, in the output image. A rendering technique is applied that samples pixel values from microimages using a thin sampling kernel, for example a thin Gaussian kernel, so that pixel values are sampled only from one or a few of the microimages.

Abstract: Various methods and apparatus for removing artifacts in frequency domain processing of light-field images are described. Methods for the reduction or removal of the artifacts are described that include methods that may be applied during frequency domain processing and a method that may be applied during post-processing of resultant angular views. The methods may be implemented in software as or in a light-field frequency domain processing module. The described methods include an oversampling method to determine the correct centers of slices, a phase multiplication method to determine the correct centers of slices, a method to exclude low-energy slices, and a cosmetic correction method.

Abstract: Method and apparatus for radiance processing by demultiplexing in the frequency domain. A frequency domain demultiplexing module obtains a radiance image captured with a lens-based radiance camera. The image includes optically mixed spatial and angular frequency components of light from a scene. The module performs frequency domain demultiplexing on the radiance image to generate multiple parallax views of the scene. The method may extract multiple slices at different angular frequencies from a Fourier transform of the radiance image, apply a Fourier transform to each of the multiple slices to generate intermediate images, stack the intermediate images to form a 3- or 4-dimensional image, apply an inverse Fourier transform along angular dimension(s) of the 3- or 4-dimensional image, and unstack the transformed 3- or 4-dimensional image to obtain the multiple parallax views. During the method, phase correction may be performed to determine the centers of the intermediate images.

Abstract: Methods and apparatus for capturing and rendering images with focused plenoptic cameras employing different filtering at different microlenses. In a focused plenoptic camera, the main lens creates an image at the focal plane. That image is re-imaged on the sensor multiple times by an array of microlenses. Different filters that provide different levels and/or types of filtering may be combined with different ones of the microlenses. A flat captured with the camera includes multiple microimages captured according to the different filters. Multiple images may be assembled from the microimages, with each image assembled from microimages captured using a different filter. A final image may be generated by appropriately combining the images assembled from the microimages. Alternatively, a final image, or multiple images, may be assembled from the microimages by first combining the microimages and then assembling the combined microimages to produce one or more output images.

Abstract: Methods and apparatus for super-resolution in integral photography are described. Several techniques are described that, alone or in combination, may improve the super-resolution process and/or the quality of super-resolved images that may be generated from flats captured with a focused plenoptic camera using a super-resolution algorithm. At least some of these techniques involve modifications to the focused plenoptic camera design. In addition, at least some of these techniques involve modifications to the super-resolution rendering algorithm. The techniques may include techniques for reducing the size of pixels, techniques for shifting pixels relative to each other so that super-resolution is achievable at more or all depths of focus, and techniques for sampling using an appropriate filter or kernel. These techniques may, for example, reduce or eliminate the need to perform deconvolution on a super-resolved image, and may improve super-resolution results and/or increase performance.

Abstract: Methods and apparatus for capturing and rendering high-quality photographs using relatively small, thin plenoptic cameras. Plenoptic camera technology, in particular focused plenoptic camera technology including but not limited to super-resolution techniques, and other technologies such as solid immersion lens (SIL) technology may be leveraged to provide thin form factor, megapixel resolution cameras suitable for use in mobile devices and other applications. In addition, at least some embodiments of these cameras may also capture radiance, allowing the imaging capabilities provided by plenoptic camera technology to be realized through appropriate rendering techniques. Hemispherical SIL technology, along with multiple main lenses and a mask on the photosensor, may be employed in some thin plenoptic cameras. Other thin cameras may include a layer between hemispherical SILs and the photosensor that effectively implements superhemispherical SIL technology in the camera.

Abstract: Methods and apparatus for capturing and rendering high-quality photographs using relatively small, thin plenoptic cameras. Plenoptic camera technology, in particular focused plenoptic camera technology including but not limited to super-resolution techniques, and other technologies such as microsphere technology may be leveraged to provide thin form factor, megapixel resolution cameras suitable for use in mobile devices and other applications. In addition, at least some embodiments of these cameras may also capture radiance, allowing the imaging capabilities provided by plenoptic camera technology to be realized through appropriate rendering techniques.

Abstract: A super-resolved demosaicing technique for rendering focused plenoptic camera data performs simultaneous super-resolution and demosaicing. The technique renders a high-resolution output image from a plurality of separate microimages in an input image at a specified depth of focus. For each point on an image plane of the output image, the technique determines a line of projection through the microimages in optical phase space according to the current point and angle of projection determined from the depth of focus. For each microimage, the technique applies a kernel centered at a position on the current microimage intersected by the line of projection to accumulate, from pixels at each microimage covered by the kernel at the respective position, values for each color channel weighted according to the kernel. A value for a pixel at the current point in the output image is computed from the accumulated values for the color channels.

Abstract: Methods, apparatus, and computer-readable storage media for calibrating focused plenoptic camera data. A calibration technique that does not modify the image data may be applied to raw plenoptic images. Calibration parameters, including but not limited to tilt angle, corner crops, main lens distance from the microlens array, sensor distance from the microlens array, and microimage size, may be specified. Calibration may include scaling down the input texture coordinates for the plenoptic image so that the new coordinate range fits the size of the texture with crops taken into account. These coordinates may be further transformed by one or more of a matrix performing a scaling, to correct for lens distortion; a rotation, to correct for tilts; and a translation that finalizes the necessary corner crops. A transformation matrix is generated that can be applied to the raw image by radiance processing techniques such as super-resolution techniques.

Abstract: Methods and apparatus for reducing plenoptic camera artifacts. A first method is based on careful design of the optical system of the focused plenoptic camera to reduce artifacts that result in differences in depth in the microimages. A second method is computational; a focused plenoptic camera rendering algorithm is provided that corrects for artifacts resulting from differences in depth in the microimages. While both the artifact-reducing focused plenoptic camera design and the artifact-reducing rendering algorithm work by themselves to reduce artifacts, the two approaches may be combined.