Abstract: One embodiment of the present invention provides a system that dynamically refocuses an image to simulate a focus plane and a depth-of-field of a virtual camera. During operation, the system receives an input image, wherein the input image includes depth information for pixels in the input image. The system also obtains parameters that specify the depth-of-field d and the location of the focus plane for the virtual camera. Next, the system uses the depth information and the parameters for the virtual camera to refocus the image. During this process, for each pixel in the input image, the system uses the depth information and the parameters for the virtual camera to determine a blur radius B for the pixel. The system then uses the blur radius B for the pixel to determine whether the pixel contributes to neighboring pixels in the refocused image.

Abstract: One embodiment of the present invention provides a plenoptic camera which captures information about the direction distribution of light rays entering the camera. Like a conventional camera, this plenoptic camera includes a main lens which receives light from objects in an object field and directs the received light onto an image plane of the camera. It also includes a photodetector array located at the image plane of the camera, which captures the received light to produce an image. However, unlike a conventional camera, the plenoptic camera additionally includes an array of optical elements located between the object field and the main lens. Each optical element in this array receives light from the object field from a different angle than the other optical elements in the array, and consequently directs a different view of the object field into the main lens. In this way, the photodetector array receives a different view of the object field from each optical element in the array.

Abstract: A method and apparatus for the block-based compression of light-field images. Light-field images may be preprocessed by a preprocessing module into a format that is compatible with the blocking scheme of a block-based compression technique, for example JPEG. The compression technique is then used to compress the preprocessed light-field images. The light-field preprocessing module reshapes the angular data in a captured light-field image into shapes compatible with the blocking scheme of the compression technique so that blocking artifacts of block-based compression are not introduced in the final compressed image. Embodiments may produce compressed 2D images for which no specific light-field image viewer is needed to preview the full light-field image. Full light-field information is contained in one compressed 2D image.

Abstract: One embodiment of the present invention relates to a system for reconstructing an image. During operation, the system receives an original image, wherein the original image includes a defective area to be reconstructed. The system also receives a reference to a texture image, which contains a texture to be used during the reconstruction. The system then divides the values of pixels in the defective area and surrounding boundary pixels, by the values of corresponding pixels in the texture image to produce a first intermediate image. Next, the system solves a partial differential equation (PDE) for non-boundary pixels in the first intermediate image subject to values of the boundary pixels in the first intermediate image to produce a second intermediate image. The system then multiplies the values of pixels in the second intermediate image by the values of corresponding pixels in the texture image to produce a replacement image.

Abstract: A method and system for generating a portion of an image uses n differential representations of an image texture, with n>1, which are mixed to produce a synthesized texture. The n differential representations are modified by weighting each differential representation using a corresponding coefficient, where a sum of squares of the coefficients is equal to a value of approximately one. The n modified differential representations are then combined to produce a combined differential representation, and the combined differential representation is integrated to obtain a modified texture region.

Abstract: Method and apparatus for full-resolution light-field capture and rendering. A radiance camera is described in which the microlenses in a microlens array are focused on the image plane of the main lens instead of on the main lens, as in conventional plenoptic cameras. The microlens array may be located at distances greater than f from the photosensor, where f is the focal length of the microlenses. Radiance cameras in which the distance of the microlens array from the photosensor is adjustable, and in which other characteristics of the camera are adjustable, are described. Digital and film embodiments of the radiance camera are described. A full-resolution light-field rendering method may be applied to light-fields captured by a radiance camera to render higher-resolution output images than are possible with conventional plenoptic cameras and rendering methods.

Abstract: Techniques for modifying an image may be applied to heal texture areas within the image. A region to be healed in an original image may be identified, and a differential representation may be calculated for at least a portion of a texture source image that provides sample texture. Samples of the texture source image differential representation may be copied to a location corresponding to the identified modification region to generate a new differential representation for the modification region. The new differential representation for the modification region may be integrated to produce a modified image. In some implementations, a differential representation may be calculated of boundary pixels that are outside of and adjacent to the region to be healed in the original image. Copying samples of the texture source image differential representation may be performed so as to obtain substantial smoothness between the copied samples and the differential boundary pixel values.

Abstract: Method and apparatus for a fast (low F/number) computational camera that incorporates two arrays of lenses. The arrays include a lenslet array in front of a photosensor and an objective lens array of two or more lenses. Each lens in the objective lens array captures light from a subject. Each lenslet in the lenslet array captures light from each objective lens and separates the captured light to project microimages corresponding to the objective lenses on a region of the photosensor under the lenslet. Thus, a plurality of microimages are projected onto and captured by the photosensor. The captured microimages may be processed in accordance with the geometry of the objective lenses to align the microimages to generate a final image. One or more other algorithms may be applied to the image data in accordance with radiance information captured by the camera, such as automatic refocusing of an out-of-focus image.

Abstract: One embodiment of the present invention provides a system that uses nonlinear filtering while resizing an image to preserve sharp detail. The system starts with an original image, which is a digital image comprised of a plurality of pixels. Next, the system resizes the original image. This involves first producing an initial resized image by using neighboring pixel values in the original image (or a conventional resizing operation) to produce initial values for subpixel locations in the original image. The system then applies a nonlinear filter using pixel values in the original and initial resized images to produce a resized output image. When this nonlinear filter updates pixel values, it weights the contributions of neighboring pixels nonlinearly.

Abstract: Systems and techniques for blending image data such that noise, texture and/or high-frequency data may be better preserved in a resulting blended image. In general, in one implementation, the techniques include blending a first image and a second image to form a blended image including a blended image component formed by a non-affine interpolation of components from the first and second images. The techniques also include blending a first image component corresponding to a first image region with a second image component corresponding to a second image region to produce a first blended component, blending a third image component corresponding to the first image region with a fourth image component corresponding to the second image region using blend coefficients, which sum to a value other than one, to produce a second blended component, and merging the first blended component with the second blended component to produce a blended image region.

Abstract: An external mask-based radiance camera may be based on an external, non-refractive mask located in front of the main camera lens. The mask modulates, but does not refract, light. The camera multiplexes radiance in the frequency domain by optically mixing different spatial and angular frequency components of light. The mask may, for example, be a mesh of opaque linear elements, which collectively form a grid, an opaque medium with transparent openings, such as circles, or a pinhole mask. Other types of masks may be used. Light may be modulated by the mask and received at the main lens of a camera. The main lens may be focused on a plane between the mask and the main lens. The received light is refracted by the main lens onto a photosensor of the camera. The photosensor may capture the received light to generate a radiance image of the scene.

Abstract: Method and apparatus for radiance processing by demultiplexing in the frequency domain. A frequency domain demultiplexing module obtains a radiance image captured with a lens-based radiance camera. The image includes optically mixed spatial and angular frequency components of light from a scene. The module performs frequency domain demultiplexing on the radiance image to generate multiple parallax views of the scene. The method may extract multiple slices at different angular frequencies from a Fourier transform of the radiance image, apply a Fourier transform to each of the multiple slices to generate intermediate images, stack the intermediate images to form a 3- or 4-dimensional image, apply an inverse Fourier transform along angular dimension(s) of the 3- or 4-dimensional image, and unstack the transformed 3- or 4-dimensional image to obtain the multiple parallax views. During the method, phase correction may be performed to determine the centers of the intermediate images.

Abstract: One embodiment of the present invention provides a system that deposterizes a posterized image. During operation, the system receives a posterized image, wherein pixels in the posterized image have color values that belong to a set of posterized color values, which is a subset of a set of possible color values for the deposterized image. Next, the system generates a deposterized image from the posterized image, wherein color values in the deposterized image are not restricted to the set of posterized color values and instead can be selected from the entire set of possible color values to reduce artifacts caused jumps in color values that arise from posterization. While generating the deposterized image, the system selects color values for pixels in the deposterized image so that the deposterized image can be reposterized to restore the original posterized image.

Abstract: Systems and techniques for blending image data such that noise, texture and/or high-frequency data may be better preserved in a resulting blended image. In general, in one implementation, the techniques include combining a first image region and a second image region to form a blended image region having a neighborhood mean and a neighborhood variance, at each of multiple points in the blended image region, that are linear blends of corresponding neighborhood means and variances from the first and second image regions using a common blending parameter.

Abstract: Methods and products are disclosed concerning extraction of selected components of a warping of an image. Given a warped image and a distortion grid, the distortion at any point in the image may be viewed locally as a displacement and a linear transform. The linear transform can be manipulated to extract elements of the local distortion such as skew, rotation, magnification and combinations. The selected components may then be selectively applied at other locations of the same or a different image, using a variety of virtual paintbrushes for different effects.

Abstract: A light field microscope incorporating a lenslet array at or near the rear aperture of the objective lens. The microscope objective lens may be supplemented with an array of low power lenslets which may be located at or near the rear aperture of the objective lens, and which slightly modify the objective lens. The result is a new type of objective lens, or an addition to existing objective lenses. The lenslet array may include, for example, 9 to 100 lenslets (small, low-power lenses with long focal lengths) that generate a corresponding number of real images. Each lenslet creates a real image on the image plane, and each image corresponds to a different viewpoint or direction of the specimen. Angular information is recorded in relations or differences among the captured real images. To retrieve this angular information, one or more of various dense correspondence techniques may be used.

Abstract: Methods and apparatus implementing systems and techniques for adjusting images. In general, in one implementation, the technique includes: receiving input defining an adjustment to be applied to a differential representation of a source image, calculating the differential representation of the source image, producing a structural representation of the source image, the structural representation corresponding to multiple types of contrast in the source image, modifying the differential representation based on the structural representation and the input defining the adjustment, and generating a modified image from the modified differential representation by solving a Poisson differential equation.

Abstract: Systems and techniques for blending image data such that noise, texture and/or high-frequency data may be better preserved in a resulting blended image. In general, in one implementation, the techniques include blending a first image and a second image to form a blended image including a blended image component formed by a non-affine interpolation of components from the first and second images. The techniques also include blending a first image component corresponding to a first image region with a second image component corresponding to a second image region to produce a first blended component, blending a third image component corresponding to the first image region with a fourth image component corresponding to the second image region using blend coefficients, which sum to a value other than one, to produce a second blended component, and merging the first blended component with the second blended component to produce a blended image region.

Abstract: A method and computer program product for generating a warping effect in an image having a plurality of regions, each associated with a distortion vector. The method includes defining a path segment vector in response to user movement of a brush cursor; modifying one or more distortion vectors having originations covered by the cursor based on the path segment vector; and repeating defining and modifying to produce a warping effect.

Abstract: Given an image with an unwanted feature, a collection of pixels (a defined region) entirely containing the unwanted feature is designated, and replacement data values for the defined region are generated, thereby healing the unwanted feature. A selected image region is healed by propagating values in a boundary region into replacement data values for the selected image region by iteratively applying a procedure or algorithm (e.g., iteratively applying one or more kernels), such that values in the boundary region are continuously put back into the iterative calculations. An example of such a function uses kernels to generate replacement data values having multiple orders of continuity at the boundary. Optionally, replacement data values for an image region to be healed are generated based on a difference between existing data values and texture data values, thereby introducing texture (e.g., pattern, noise) components to a resulting healed region in the image.