Abstract: A plenoptic camera captures a plenoptic image of an object illuminated by a point source (preferably, collimated illumination). The plenoptic image is a sampling of the four-dimensional light field reflected from the object. The plenoptic image is made up of superpixels, each of which is made up of subpixels. Each superpixel captures light from a certain region of the object (i.e., a range of x,y spatial locations) and the subpixels within a superpixel capture light propagating within a certain range of directions (i.e., a range of u,v spatial directions). Accordingly, optical properties estimation, surface normal reconstruction, depth estimation, and three-dimensional rendering can be provided by processing only a single plenoptic image. In one approach, the plenoptic image is used to estimate the bidirectional reflectance distribution function (BRDF) of the object surface.

Abstract: A mobile information gateway comprises: a wearable human interface module having an image delivery and display mechanism for presenting information overlaid upon a wide field of view, a computing and communication module adapted to send and receive information to and from the human interface module; and a backend service server coupled for processing data. The present invention also includes a method for using a mobile information gateway comprises: capturing information with a mobile information gateway device; processing the information captured by the mobile information gateway device to detect an emergency condition; analyzing the information captured by the mobile information gateway device to identify an emergency response; retrieving instructional information corresponding the emergency response; and providing the instructional information on the mobile information gateway device.

Abstract: A scene is segmented into objects based on light field data for the scene, including based on image pixel values (e.g., intensity) and disparity map(s). In one aspect, the light field data is used to estimate one or more disparity maps for the scene taken from different viewpoints. The scene is then segmented into a plurality of regions that correspond to objects in the scene. Unlike other approaches, the regions can be variable-depth. In one approach, the regions are defined by boundaries. The boundaries are determined by varying the boundary to optimize an objective function for the region defined by the boundary. The objective function is based in part on a similarity function that measures a similarity of image pixel values for pixels within the boundary and also measures a similarity of disparities for pixels within the boundary.

Abstract: A procedure to calibrate a depth-disparity mapping for a plenoptic imaging system. In one aspect, one or more test objects located at known field positions and known depths are presented to the plenoptic imaging system. The plenoptic imaging system captures plenoptic images of the test objects. The plenoptic images include multiple images of the test objects captured from different viewpoints. Disparities for the test objects are calculated based on the multiple images taken from the different viewpoints. Since the field positions and depths of the test objects are known, a mapping between depth and disparity as a function of field position can be determined.

Abstract: A system and method for facilitating private user interaction is disclosed. The method comprises capturing information with a first mobile information gateway device; processing the captured information to determine an identity of a first user; processing the captured information to authenticate the first user; processing the identity of the first user to retrieve information of the first user; and presenting the retrieved information of the first user overlaid upon a first field of view by the first mobile information gateway device; and presenting the retrieved information to a second user overlaid upon a second field of view by the second mobile information gateway device.

Abstract: Designs for a light field otoscope are disclosed. An example light field otoscope includes an objective lens group, relay optics and a plenoptic sensor (e.g., microlens array and sensor array). The objective lens group images an interior of a human ear and is characterized by a pupil plane and an image plane. The relay optics is positioned between the objective lens group and the plenoptic sensor. It relays the image plane to the microlens array and relays the pupil plane to the sensor array.

Abstract: Designs for a light field otoscope are disclosed. An example light field otoscope includes an objective lens group, relay optics and a plenoptic sensor (e.g., microlens array and sensor array). The objective lens group images an interior of a human ear and is characterized by a pupil plane and an image plane. The relay optics is positioned between the objective lens group and the plenoptic sensor. It relays the image plane to the microlens array and relays the pupil plane to the sensor array.

Abstract: A single imaging platform for making in vivo measurements of an individual's eye, where the measurements are sufficient to construct an optical model of the individual's eye. The platform includes a plenoptic opthalmic camera and an illumination module. In one configuration, the plenoptic opthalmic camera captures a plenoptic image of a corneal anterior surface of the individual's eye. In another configuration, the plenoptic opthalmic camera operates as a wavefront sensor to measure a wavefront produced by the individual's eye.

Abstract: A plenoptic camera captures a plenoptic image of an object illuminated by a point source (preferably, collimated illumination). The plenoptic image is a sampling of the four-dimensional light field reflected from the object. The plenoptic image is made up of superpixels, each of which is made up of subpixels. Each superpixel captures light from a certain region of the object (i.e., a range of x,y spatial locations) and the subpixels within a superpixel capture light propagating within a certain range of directions (i.e., a range of u,v spatial directions). Accordingly, optical properties estimation, surface normal reconstruction, depth estimation, and three-dimensional rendering can be provided by processing only a single plenoptic image. In one approach, the plenoptic image is used to estimate the bidirectional reflectance distribution function (BRDF) of the object surface.

Abstract: Multiframe reconstruction combines a set of acquired images into a reconstructed image. Here, which images to acquire are selected based at least in part on the content of previously acquired images. In one approach, a set of at least three images of an object are acquired at different acquisition settings. For at least one of the images in the set, the acquisition setting for the image is determined based at least in part on the content of previously acquired images. Multiframe image reconstruction, preferably via a multi-focal display, is applied to the set of acquired images to synthesize a reconstructed image of the object.

Abstract: Light field images of a three-dimensional scene are transformed from an (image,view) domain to an (image,scale,depth) domain. Processing then occurs in the (image,scale,depth) domain.

Abstract: A multi-focal display represents a 3-dimensional scene by a series of 2-dimensional images located at different focal planes. The locations of the focal planes are selected based on an analysis of the three-dimensional scene to be rendered by the multi-focal display.

Abstract: A multifocal display for rendering a 3D scene as a series of 2D images. In one aspect, the multifocal display includes a display, an optical imaging system, a refractive focus actuator and a controller. The display renders the 2D images. The optical imaging system is image-side telecentric and creates an image of the display. The refractive focus actuator is positioned at the pupil of the optical imaging system. Thus, adjusting the refractive focus actuator alters a location of the image of the display but does not significantly alter a size of the image. The controller coordinates adjustment of the refractive focus actuator with rendering of the 2D images on the display. The waveform driving the focus actuator is preferably designed to reduce ringing and jitter effects.

Abstract: A mobile information gateway comprises a wearable human interface module having an image delivery and display mechanism for presenting information overlaid upon a wide field of view, a computing and communication module adapted receive information from the human interface module and adapted to send commands and information to the human interface module including information for presentation; and a backend service server coupled for processing data from the computing and communication module including user identification and verification.

Abstract: The disclosure includes a system and method for calibrating a binocular optical see-through augmented reality display. A calibration application renders one or more images of a virtual object on the one or more displays of the human interface device, and receives a user input to adjust a position of the one or more images on the one or more displays. In response to the input, the calibration application identifies a first and second boundary of a target range surrounding a real-world object at a first and second depth, respectively. The calibration application determines a first and second disparity corresponding to the first and second boundary and may record the disparities in relation to the depth of the real-world object.

Abstract: The pupil image function (PIF) matrix of a plenoptic imaging system is calibrated, taking advantage of the low rank of the PIF matrix. In one approach, the low rank is utilized by identifying a subspace for the PIF matrix and then estimating the PIF matrix within that subspace. This can lead to a significant reduction in the number of calibration patterns used to estimate the PIF matrix.

Abstract: The disclosure includes a system and method for calibrating a binocular optical see-through augmented reality display. A calibration application renders a virtual object to overlay and align with a real-world object on the display, receives a depth between the real-world object and the eyes of the user, receives user input to move the virtual object on the display for depth-disparity calibration, and determines a mapping between the depth and the disparity.

Abstract: A three-dimensional scene is modeled as a set of layers representing different depths of the scene, and as masks representing occlusions of layers by other layers. The layers are represented as linear combinations of atoms selected from an overcomplete dictionary. An iterative approach is used to alternately estimate the atom coefficients for layers from a light field image of the scene, assuming values for the masks, and to estimate the masks given the estimated layers. In one approach, the atoms in the dictionary are ridgelets oriented at different angles, where there is a correspondence between depth and angle.

Abstract: A method for designing a color filter module used in a plenoptic XYZ imaging system. In one approach, the color filter module is spatially partitioned into filter cells, and the spatial partition is designed by considering data captured at the sensor in light of variations in ?E.

Abstract: Light field images of a three-dimensional scene are transformed from an (image,view) domain to an (image,scale,depth) domain. Processing then occurs in the (image,scale,depth) domain.