Abstract: Methods and apparatus for reducing plenoptic camera artifacts. A first method is based on careful design of the optical system of the focused plenoptic camera to reduce artifacts that result in differences in depth in the microimages. A second method is computational; a focused plenoptic camera rendering algorithm is provided that corrects for artifacts resulting from differences in depth in the microimages. While both the artifact-reducing focused plenoptic camera design and the artifact-reducing rendering algorithm work by themselves to reduce artifacts, the two approaches may be combined.

Abstract: Method and apparatus for full-resolution light-field capture and rendering. A radiance camera is described in which the microlenses in a microlens array are focused on the image plane of the main lens instead of on the main lens, as in conventional plenoptic cameras. The microlens array may be located at distances greater than f from the photosensor, where f is the focal length of the microlenses. Radiance cameras in which the distance of the microlens array from the photosensor is adjustable, and in which other characteristics of the camera are adjustable, are described. Digital and film embodiments of the radiance camera are described. A full-resolution light-field rendering method may be applied to flats captured by a radiance camera to render higher-resolution output images than are possible with conventional plenoptic cameras and rendering methods.

Abstract: Method and apparatus for full-resolution light-field capture and rendering. A radiance camera is described in which the microlenses in a microlens array are focused on the image plane of the main lens instead of on the main lens, as in conventional plenoptic cameras. The microlens array may be located at distances greater than f from the photosensor, where f is the focal length of the microlenses. Radiance cameras in which the distance of the microlens array from the photosensor is adjustable, and in which other characteristics of the camera are adjustable, are described. Digital and film embodiments of the radiance camera are described. A full-resolution light-field rendering method may be applied to light-fields captured by a radiance camera to render higher-resolution output images than are possible with conventional plenoptic cameras and rendering methods.

Abstract: A method and apparatus for the block-based compression of light-field images. Light-field images may be preprocessed by a preprocessing module into a format that is compatible with the blocking scheme of a block-based compression technique, for example JPEG. The compression technique is then used to compress the preprocessed light-field images. The light-field preprocessing module reshapes the angular data in a captured light-field image into shapes compatible with the blocking scheme of the compression technique so that blocking artifacts of block-based compression are not introduced in the final compressed image. Embodiments may produce compressed 2D images for which no specific light-field image viewer is needed to preview the full light-field image. Full light-field information is contained in one compressed 2D image.

Abstract: An external mask-based radiance camera may be based on an external, non-refractive mask located in front of the main camera lens. The mask modulates, but does not refract, light. The camera multiplexes radiance in the frequency domain by optically mixing different spatial and angular frequency components of light. The mask may, for example, be a mesh of opaque linear elements, which collectively form a grid, an opaque medium with transparent openings, such as circles, or a pinhole mask. Other types of masks may be used. Light may be modulated by the mask and received at the main lens of a camera. The main lens may be focused on a plane between the mask and the main lens. The received light is refracted by the main lens onto a photosensor of the camera. The photosensor may capture the received light to generate a radiance image of the scene.

Abstract: An external mask-based radiance camera may be based on an external, non-refractive mask located in front of the main camera lens. The mask modulates, but does not refract, light. The camera multiplexes radiance in the frequency domain by optically mixing different spatial and angular frequency components of light. The mask may, for example, be a mesh of opaque linear elements, which collectively form a grid, an opaque medium with transparent openings, such as circles, or a pinhole mask. Other types of masks may be used. Light may be modulated by the mask and received at the main lens of a camera. The main lens may be focused on a plane between the mask and the main lens. The received light is refracted by the main lens onto a photosensor of the camera. The photosensor may capture the received light to generate a radiance image of the scene.

Abstract: Techniques for modifying an image may be applied to heal texture areas within the image. A region to be healed in an original image may be identified, and a differential representation may be calculated for at least a portion of a texture source image that provides sample texture. Samples of the texture source image differential representation may be copied to a location corresponding to the identified modification region to generate a new differential representation for the modification region. The new differential representation for the modification region may be integrated to produce a modified image. In some implementations, a differential representation may be calculated of boundary pixels that are outside of and adjacent to the region to be healed in the original image. Copying samples of the texture source image differential representation may be performed so as to obtain substantial smoothness between the copied samples and the differential boundary pixel values.

Abstract: An external mask-based radiance camera may be based on an external, non-refractive mask located in front of the main camera lens. The mask modulates, but does not refract, light. The camera multiplexes radiance in the frequency domain by optically mixing different spatial and angular frequency components of light. The mask may, for example, be a mesh of opaque linear elements, which collectively form a grid, an opaque medium with transparent openings, such as circles, or a pinhole mask. Other types of masks may be used. Light may be modulated by the mask and received at the main lens of a camera. The main lens may be focused on a plane between the mask and the main lens. The received light is refracted by the main lens onto a photosensor of the camera. The photosensor may capture the received light to generate a radiance image of the scene.

Abstract: Method and apparatus for full-resolution light-field capture and rendering. A radiance camera is described in which the microlenses in a microlens array are focused on the image plane of the main lens instead of on the main lens, as in conventional plenoptic cameras. The microlens array may be located at distances greater than f from the photosensor, where f is the focal length of the microlenses. Radiance cameras in which the distance of the microlens array from the photosensor is adjustable, and in which other characteristics of the camera are adjustable, are described. Digital and film embodiments of the radiance camera are described. A full-resolution light-field rendering method may be applied to light-fields captured by a radiance camera to render higher-resolution output images than are possible with conventional plenoptic cameras and rendering methods.

Abstract: Method and apparatus for full-resolution light-field capture and rendering. A radiance camera is described in which the microlenses in a microlens array are focused on the image plane of the main lens instead of on the main lens, as in conventional plenoptic cameras. The microlens array may be located at distances greater than f from the photosensor, where f is the focal length of the microlenses. Radiance cameras in which the distance of the microlens array from the photosensor is adjustable, and in which other characteristics of the camera are adjustable, are described. Digital and film embodiments of the radiance camera are described. A full-resolution light-field rendering method may be applied to light-fields captured by a radiance camera to render higher-resolution output images than are possible with conventional plenoptic cameras and rendering methods.

Abstract: One embodiment of the present invention provides a system that dynamically refocuses an image to simulate a focus plane and a depth-of-field of a virtual camera. During operation, the system receives an input image, wherein the input image includes depth information for pixels in the input image. The system also obtains parameters that specify the depth-of-field d and the location of the focus plane for the virtual camera. Next, the system uses the depth information and the parameters for the virtual camera to refocus the image. During this process, for each pixel in the input image, the system uses the depth information and the parameters for the virtual camera to determine a blur radius B for the pixel. The system then uses the blur radius B for the pixel to determine whether the pixel contributes to neighboring pixels in the refocused image.

Abstract: Method and apparatus for a fast (low F/number) computational camera that incorporates two arrays of lenses. The arrays include a lenslet array in front of a photosensor and an objective lens array of two or more lenses. Each lens in the objective lens array captures light from a subject. Each lenslet in the lenslet array captures light from each objective lens and separates the captured light to project microimages corresponding to the objective lenses on a region of the photosensor under the lenslet. Thus, a plurality of microimages are projected onto and captured by the photosensor. The captured microimages may be processed in accordance with the geometry of the objective lenses to align the microimages to generate a final image. One or more other algorithms may be applied to the image data in accordance with radiance information captured by the camera, such as automatic refocusing of an out-of-focus image.

Abstract: Methods and apparatus for light-field capture with large depth of field. A design methodology is described in which the relationships among various plenoptic camera parameters, including inverse magnification, F-number, focal length, wavelength, and pixel size, may be analyzed to design plenoptic cameras that provide increased depth-of-field when compared to conventional plenoptic cameras. Plenoptic cameras are described, which may be implemented according to the design methodology, and in which both Keplerian telescopic and Galilean telescopic imaging can be realized at the same time while providing a larger depth of field than is realized in conventional plenoptic cameras, thus capturing light-field images that capture “both sides” in which all but a small region of the scene is in focus. In some embodiments, apertures may be added to the microlenses so that depth of field is increased.

Abstract: A light field microscope incorporating a lenslet array at or near the rear aperture of the objective lens. The microscope objective lens may be supplemented with an array of low power lenslets which may be located at or near the rear aperture of the objective lens, and which slightly modify the objective lens. The result is a new type of objective lens, or an addition to existing objective lenses. The lenslet array may include, for example, 9 to 100 lenslets (small, low-power lenses with long focal lengths) that generate a corresponding number of real images. Each lenslet creates a real image on the image plane, and each image corresponds to a different viewpoint or direction of the specimen. Angular information is recorded in relations or differences among the captured real images. To retrieve this angular information, one or more of various dense correspondence techniques may be used.

Abstract: One embodiment of the present invention provides a system that uses nonlinear filtering while resizing an image to preserve sharp detail. The system starts with an original image, which is a digital image comprised of a plurality of pixels. Next, the system resizes the original image. This involves first producing an initial resized image by using neighboring pixel values in the original image (or a conventional resizing operation) to produce initial values for subpixel locations in the original image. The system then applies a nonlinear filter using pixel values in the original and initial resized images to produce a resized output image. When this nonlinear filter updates pixel values, it weights the contributions of neighboring pixels nonlinearly.

Abstract: Methods and products are disclosed concerning extraction of selected components of a warping of an image. Given a warped image and a distortion grid, the distortion at any point in the image may be viewed locally as a displacement and a linear transform. The linear transform can be manipulated to extract elements of the local distortion such as skew, rotation, magnification and combinations. The selected components may then be selectively applied at other locations of the same or a different image, using a variety of virtual paintbrushes for different effects.

Abstract: Methods and apparatus implementing systems and techniques for adjusting images. In general, in one implementation, the technique includes: receiving input defining an adjustment to be applied to a differential representation of a source image, calculating the differential representation of the source image, producing a structural representation of the source image, the structural representation corresponding to multiple types of contrast in the source image, modifying the differential representation based on the structural representation and the input defining the adjustment, and generating a modified image from the modified differential representation by solving a Poisson differential equation.

Abstract: One embodiment of the present invention provides a plenoptic camera which captures information about the direction distribution of light rays entering the camera. Like a conventional camera, this plenoptic camera includes a main lens which receives light from objects in an object field and directs the received light onto an image plane of the camera. It also includes a photodetector array located at the image plane of the camera, which captures the received light to produce an image. However, unlike a conventional camera, the plenoptic camera additionally includes an array of optical elements located between the object field and the main lens. Each optical element in this array receives light from the object field from a different angle than the other optical elements in the array, and consequently directs a different view of the object field into the main lens. In this way, the photodetector array receives a different view of the object field from each optical element in the array.

Abstract: One embodiment of the present invention provides a system that deposterizes a posterized image. During operation, the system receives a posterized image, wherein pixels in the posterized image have color values that belong to a set of posterized color values, which is a subset of a set of possible color values for the deposterized image. Next, the system generates a deposterized image from the posterized image, wherein color values in the deposterized image are not restricted to the set of posterized color values and instead can be selected from the entire set of possible color values to reduce artifacts caused jumps in color values that arise from posterization. While generating the deposterized image, the system selects color values for pixels in the deposterized image so that the deposterized image can be reposterized to restore the original posterized image.

Abstract: Method and apparatus for full-resolution light-field capture and rendering. A radiance camera is described in which the microlenses in a microlens array are focused on the image plane of the main lens instead of on the main lens, as in conventional plenoptic cameras. The microlens array may be located at distances greater than f from the photosensor, where f is the focal length of the microlenses. Radiance cameras in which the distance of the microlens array from the photosensor is adjustable, and in which other characteristics of the camera are adjustable, are described. Digital and film embodiments of the radiance camera are described. A full-resolution light-field rendering method may be applied to flats captured by a radiance camera to render higher-resolution output images than are possible with conventional plenoptic cameras and rendering methods.