Abstract: A method and apparatus for optical field communication, wherein incident light is spread on the surface of an image sensor with a diffuser element; a conventional digital image is captured with high exposure pixel rows of the image sensor; and the light intensity on two successive low exposure pixel rows of the image sensor is recorded. The recorded light intensities of the two successive low exposure pixel rows are compared; and in response to comparing the recorded light intensities of the two successive low exposure pixel rows, a value of a bit received via optical field communication is determined.

Abstract: In exemplary implementations of this invention, a light source illuminates a scene and a light sensor captures data about light that scatters from the scene. The light source emits multiple modulation frequencies, either in a temporal sequence or as a superposition of modulation frequencies. Reference signals that differ in phase are applied to respective subregions of each respective pixel. The number of subregions per pixel, and the number of reference signals per pixel, is preferably greater than four. One or more processors calculate a full cross-correlation function for each respective pixel, by fitting light intensity measurements to a curve, the light intensity measurements being taken, respectively, by respective subregions of the respective pixel. The light sensor comprises M subregions. A lenslet is placed over each subregion, so that each subregion images the entire scene. At least one temporal sequence of frames is taken, one frame per subregion.

Abstract: In exemplary implementations of this invention, a camera can capture multiple millions of frames per second, such that each frame is 2D image, rather than a streak. A light source in the camera emits ultrashort pulses of light to illuminate a scene. Scattered light from the scene returns to the camera. This incoming light strikes a photocathode, which emits electrons, which are detected by a set of phosphor blocks, which emit light, which is detected by a light sensor. Voltage is applied to plates to create an electric field that deflects the electrons. The voltage varies in a temporal “stepladder” pattern, deflecting the electrons by different amounts, such that the electrons hit different phosphor blocks at different times during the sequence. Each phosphor block (together with the light sensor) captures a separate frame in the sequence. A mask may be used to increase resolution.

Abstract: An active imaging system, which includes a light source and light sensor, generates structured illumination. The light sensor captures transient light response data regarding reflections of light emitted by the light source. The transient light response data is wavelength-resolved. One or more processors process the transient light response data and data regarding the structured illumination to calculate a reflectance spectra map of an occluded surface. The processors also compute a 3D geometry of the occluded surface.

Abstract: In exemplary implementations of this invention, a multi-frequency ToF camera mitigates the effect of multi-path interference (MPI), and can calculate an accurate depth map despite MPI. A light source in the multi-frequency camera emits light in a temporal sequence of different frequencies. For example, the light source can emit a sequence of ten equidistant frequencies f=10 MHz, 20 MHz, 30 MHz, . . . , 100 MHz. At each frequency, a lock-in sensor within the ToF camera captures 4 frames. From these 4 frames, one or more processors compute, for each pixel in the sensor, a single complex number. The processors stack all of such complex quantities (one such complex number per pixel per frequency) and solve for the depth and intensity, using a spectral estimation technique.

Abstract: In exemplary implementations of this invention, light from a light field projector is transmitted through an angle-expanding screen to create a glasses-free, 3D display. The display can be horizontal-only parallax or full parallax. In the former case, a vertical diffuser may positioned in the optical stack. The angle-expanding screen may comprise two planar arrays of optical elements (e.g., lenslets or lenticules) separated from each other by the sum of their focal distances. Alternatively, a light field projector may project light rays through a focusing lens onto a diffuse, transmissive screen. In this alternative approach, the light field projector may comprise two spatial light modulators (SLMs). A focused image of the first SLM, and a slightly blurred image of the second SLM, are optically combined on the diffuser, creating a combined image that has a higher spatial resolution and a higher dynamic range than either of two SLMs.

Abstract: In exemplary implementations of this invention, an automultiscopic display device includes (1) one or more spatially addressable, light attenuating layers, and (2) a controller which is configured to perform calculations to control the device. In these calculations, tensors provide sparse, memory-efficient representations of a light field. The calculations include using weighted nonnegative tensor factorization (NTF) to solve an optimization problem. The NTF calculations can be sufficiently efficient to achieve interactive refresh rates. Either a directional backlight or a uniform backlight may be used. For example, the device may have (1) a high resolution LCD in front, and (2) a low resolution directional backlight. Or, for example, the device may have a uniform backlight and three or more LCD panels. In these examples, all of the LCDs and the directional backlight (if applicable) may be time-multiplexed.

Abstract: In exemplary implementations of this invention, a light field camera uses a light field dictionary to reconstruct a 4D light field from a single photograph. The light field includes both angular and spatial information and has a spatial resolution equal to the spatial resolution of the imaging sensor. Light from a scene passes through a coded spatial light modulator (SLM) before reaching an imaging sensor. Computer processors reconstruct a light field. This reconstruction includes computing a sparse or compressible coefficient vector using a light field dictionary matrix. Each column vector of the dictionary matrix is a light field atom. These light field atoms each, respectively, comprise information about a small 4D region of a light field. Reconstruction quality may be improved by using an SLM that is as orthogonal as possible to the dictionary.

Abstract: In exemplary implementations, this invention comprises apparatus for retinal self-imaging. Visual stimuli help the user self-align his eye with a camera. Bi-ocular coupling induces the test eye to rotate into different positions. As the test eye rotates, a video is captured of different areas of the retina. Computational photography methods process this video into a mosaiced image of a large area of the retina. An LED is pressed against the skin near the eye, to provide indirect, diffuse illumination of the retina. The camera has a wide field of view, and can image part of the retina even when the eye is off-axis (when the eye's pupillary axis and camera's optical axis are not aligned). Alternately, the retina is illuminated directly through the pupil, and different parts of a large lens are used to image different parts of the retina. Alternately, a plenoptic camera is used for retinal imaging.

Abstract: In exemplary implementations, this invention is a tool for subjective assessment of the visual acuity of a human eye. A microlens or pinhole array is placed over a high-resolution display. The eye is brought very near to the device. Patterns are displayed on the screen under some of the lenslets or pinholes. Using interactive software, a user causes the patterns that the eye sees to appear to be aligned. The software allows the user to move the apparent position of the patterns. This apparent motion is achieved by pre-warping the position and angle of the ray-bundles exiting the lenslet display. As the user aligns the apparent position of the patterns, the amount of pre-warping varies. The amount of pre-warping required in order for the user to see what appears to be a single, aligned pattern indicates the lens aberration of the eye.

Abstract: In illustrative implementations of this invention, multi-path analysis of transient illumination is used to reconstruct scene geometry, even of objects that are occluded from the camera. An ultrafast camera system is used. It comprises a photo-sensor (e.g., accurate in the picosecond range), a pulsed illumination source (e.g. a femtosecond laser) and a processor. The camera emits a very brief light pulse that strikes a surface and bounces. Depending on the path taken, part of the light may return to the camera after one, two, three or more bounces. The photo-sensor captures the returning light bounces in a three-dimensional time image I(x,y,t) for each pixel. The camera takes different angular samples from the same viewpoint, recording a five-dimensional STIR (Space Time Impulse Response). A processor analyzes onset information in the STIR to estimate pairwise distances between patches in the scene, and then employs isometric embedding to estimate patch coordinates.

Abstract: In exemplary implementations of this invention, cataracts in the human eye are assessed and mapped by measuring the perceptual impact of forward scattering on the foveal region. The same method can be used to measure scattering/blocking media inside lenses of a camera. Close-range anisotropic displays create collimated beams of light to scan through sub-apertures, scattering light as it strikes a cataract. User feedback is accepted and analyzed, to generate maps for opacity, attenuation, contrast and sub-aperture point-spread functions (PSFs). Optionally, the PSF data is used to reconstruct the individual's cataract-affected view.

Abstract: In exemplary implementations of this invention, an automultiscopic display device includes (1) one or more spatially addressable, light attenuating layers, and (2) a controller which is configured to perform calculations to control the device. In these calculations, tensors provide sparse, memory-efficient representations of a light field. The calculations include using weighted nonnegative tensor factorization (NTF) to solve an optimization problem. The NTF calculations can be sufficiently efficient to achieve interactive refresh rates. Either a directional backlight or a uniform backlight may be used. For example, the device may have (1) a high resolution LCD in front, and (2) a low resolution directional backlight. Or, for example, the device may have a uniform backlight and three or more LCD panels. In these examples, all of the LCDs and the directional backlight (if applicable) may be time-multiplexed.

Abstract: In exemplary implementations of this invention, a flat screen device displays a 3D scene. The 3D display may be viewed by a person who is not wearing any special glasses. The flat screen device displays dynamically changing 3D imagery, with a refresh rate so fast that the device may be used for 3D movies or for interactive, 3D display. The flat screen device comprises a stack of LCD layers with two crossed polarization filters, one filter at each end of the stack. One or more processors control the voltage at each pixel of each LCD layer, in order to control the polarization state rotation induced in light at that pixel. The processor employs an algorithm that models each LCD layer as a spatially-controllable polarization rotator, rather than a conventional spatial light modulator that directly attenuates light. Color display is achieved using field sequential color illumination with monochromatic LCDs.

Abstract: A bidirectional screen alternately switches between a display mode showing conventional graphics and a capture mode in which the LCD backlight is disabled and the LCD displays a pinhole array or a tiled-broadband code. A large-format image sensor is placed behind the liquid crystal layer. Together, the image sensor and LCD function as a mask-based light field camera, capturing an array of images equivalent to that produced by an array of cameras spanning the display surface. The recovered multi-view orthographic imagery is used to passively estimate the depth of scene points from focus.

Abstract: In an exemplary implementation of this invention, light from a scattering scene passes through a spatial light attenuation pattern and strikes a sensor plane of a camera. Based on said camera's measurements of the received light, a processing unit calculates angular samples of the received light. Light that strikes the sensor plane at certain angles comprises both scattered and directly transmitted components; whereas light that strikes at other angles comprises solely scattered light. A processing unit calculates a polynomial model for the intensity of scattered-only light that falls at the latter angles, and further estimates the direct-only component of the light that falls at the former angles. Further, a processing unit may use the estimated direct component to calculate a reconstructed 3D shape, such as a 3D shape of a finger vein pattern, using an algebraic reconstruction technique.

Abstract: In exemplary implementations, this invention comprises apparatus for retinal self-imaging. Visual stimuli help the user self-align his eye with a camera. Bi-ocular coupling induces the test eye to rotate into different positions. As the test eye rotates, a video is captured of different areas of the retina. Computational photography methods process this video into a mosaiced image of a large area of the retina. An LED is pressed against the skin near the eye, to provide indirect, diffuse illumination of the retina. The camera has a wide field of view, and can image part of the retina even when the eye is off-axis (when the eye's pupillary axis and camera's optical axis are not aligned). Alternately, the retina is illuminated directly through the pupil, and different parts of a large lens are used to image different parts of the retina. Alternately, a plenoptic camera is used for retinal imaging.

Abstract: A single camera acquires an input image of a scene as observed in an array of spheres, wherein pixels in the input image corresponding to each sphere form a sphere image. A set of virtual cameras are defined for each sphere on a line joining a center of the sphere and a center of projection of the camera, wherein each virtual camera has a different virtual viewpoint and an associated cone of rays, appearing as a circle of pixels on its virtual image plane. A projective texture mapping of each sphere image is applied to all of the virtual cameras on the virtual image plane to produce a virtual camera image comprising circle of pixels. Each virtual camera image for each sphere is then projected to a refocusing geometry using a refocus viewpoint to produce a wide-angle lightfield view, which are averaged to produce a refocused wide-angle image.

Abstract: In exemplary implementations of this invention, a flat screen device displays a 3D scene. The 3D display may be viewed by a person who is not wearing any special glasses. The flat screen device displays dynamically changing 3D imagery, with a refresh rate so fast that the device may be used for 3D movies or for interactive, 3D display. The flat screen device comprises a stack of LCD layers with two crossed polarization filters, one filter at each end of the stack. One or more processors control the voltage at each pixel of each LCD layer, in order to control the polarization state rotation induced in light at that pixel. The processor employs an algorithm that models each LCD layer as a spatially-controllable polarization rotator, rather than a conventional spatial light modulator that directly attenuates light. Color display is achieved using field sequential color illumination with monochromatic LCDs.

Abstract: In exemplary implementations of this invention, a set of two scanning mirrors scans the one dimensional field of view of a streak camera across a scene. The mirrors are continuously moving while the camera takes streak images. Alternately, the mirrors may only between image captures. An illumination source or other captured event is synchronized with the camera so that for every streak image the scene looks different. The scanning assures that different parts of the scene are captured.