Abstract: This disclosure describes a novel lens system having an entrance pupil at a location matching, or nearly matching, the eye's optical system entrance pupil. The described lens system is significantly thinner than a single-aperture lens with matched entrance pupil and is achieved by using an array of camera elements. Each camera element contains camera optics called “lenslets” and a sensor (e.g., an image or depth sensor), where a lenslet can include one or more lens elements (e.g., a compound lens). Individual camera elements can be arranged such that THEIR optical axes intersect at, or near, the eye's entrance pupil location. In addition, each camera element's field-of-view (FOV) corresponds to a small sector of a large FOV. The FOVs of adjacent camera elements can be non- or slightly-overlapping so that a wide-angle image can be formed by concatenation of the individual camera element images.

Abstract: In one implementation, an apparatus includes a display to emit light in a first wavelength range, a camera to detect light in a second wavelength range, an eyepiece to distort light in the first wavelength range, and one or more light sources, disposed between the eyepiece and the display, to emit light in the second wavelength range. In one implementation, an apparatus includes a display to emit light in a first wavelength range, one or more light sources to emit light in a second wavelength range, a camera to detect light in a second wavelength range, and an eyepiece to reflect and refract light in the first wavelength range while passing, without substantial distortion, light in the second wavelength range.

Abstract: In one implementation, an apparatus includes a display having a front surface and a back surface. The display includes a plurality of pixel regions that emit light from the front surface to display a displayed image and a plurality of apertures that transmit light from the front surface to the back surface. The apparatus includes a camera disposed on a side of the back surface of the display. The camera is configured to capture a captured image. The apparatus includes a processor coupled to the display and the camera. The processor is configured to receive the captured image and apply a first digital filter to a first portion of the captured image and a second digital filter, different than the first digital filter, to a second portion of the captured image to reduce image distortion caused by the display.

Abstract: An apparatus for touch-sensing includes a light-emitting layer covered by a transparent layer and configured to illuminate a surface touching the transparent layer and to allow transmission of reflected light rays from the surface to underlying layers. The underlying layers include an optical coupling layer, a collimator layer and a pixelated image sensor. The optical coupling layer bends the reflected light rays to create oblique light rays. The collimator layer includes a number of apertures to collimate the received light rays. The pixelated image sensor senses the collimated oblique light rays.

Abstract: A device for MR/VR systems that includes a two-dimensional array of cameras that capture images of respective portions of a scene. The cameras are positioned along a spherical surface so that the cameras have adjacent fields of view. The entrance pupils of the cameras are positioned at or near the user's eye while the cameras also form optimized images at the sensor. Methods for reducing the number of cameras in an array, as well as methods for reducing the number of pixels read from the array and processed by the pipeline, are also described.

Abstract: This disclosure describes a novel lens system having an entrance pupil at a location matching, or nearly matching, the eye's optical system entrance pupil. The described lens system is significantly thinner than a single-aperture lens with matched entrance pupil and is achieved by using an array of camera elements. Each camera element contains camera optics called “lenslets” and a sensor (e.g., an image or depth sensor), where a lenslet can include one or more lens elements (e.g., a compound lens). Individual camera elements can be arranged such that THEIR optical axes intersect at, or near, the eye's entrance pupil location. In addition, each camera element's field-of-view (FOV) corresponds to a small sector of a large FOV. The FOVs of adjacent camera elements can be non- or slightly-overlapping so that a wide-angle image can be formed by concatenation of the individual camera element images.

Abstract: An apparatus for touch-sensing includes a light-emitting layer covered by a transparent layer and configured to illuminate a surface touching the transparent layer and to allow transmission of reflected light rays from the surface to underlying layers. The underlying layers include an optical coupling layer, a collimator layer and a pixelated image sensor. The optical coupling layer bends the reflected light rays to create oblique light rays. The collimator layer includes a number of apertures to collimate the received light rays. The pixelated image sensor senses the collimated oblique light rays.

Abstract: A system and method for determining an object or product represented in an image is disclosed. The system receives a first image, determines a region of interest in the first image, determines a classification score for the region of interest using a convolutional neural network that assigns the region of interest the classification score corresponding to a class, and identifies a first product in the first image based on the classification score.

Abstract: An eye tracking system for detecting position and movements of a user's eyes in a head-mounted display (HMD). The eye tracking system includes at least one eye tracking camera, an illumination source that emits infrared light towards the user's eyes, and diffraction gratings located at the eyepieces. The diffraction gratings redirect or reflect at least a portion of infrared light reflected off the user's eyes, while allowing visible light to pass. The cameras capture images of the user's eyes from the infrared light that is redirected or reflected by the diffraction gratings.

Abstract: An eye tracking system for detecting position and movements of a user's eyes in a head-mounted display (HMD). The eye tracking system may include at least one near-IR (NIR) eye tracking camera positioned at each side of the user's face and pointed towards eye-facing surfaces of eyepieces of the HMD, an illumination source that emits NIR light towards the user's eyes, and hot mirrors located between the eye-facing surfaces of the eyepieces and the user's eyes. The hot mirrors reflect at least a portion of NIR light, while allowing visible light to pass. The cameras capture images of the user's eyes reflected by the hot mirrors. Positioning the hot mirrors at or near the eye-facing surfaces of the eyepieces allows the cameras to be placed at the sides of the user's face without having to image through the eyepieces.

Abstract: A plenoptic otoscope enables three-dimensional and/or spectral imaging of the inside of the ear to assist in improved diagnosis of inflammations and infections. The plenoptic otoscope includes a primary imaging system and a plenoptic sensor. The primary imaging system includes an otoscope objective and relay optics, which cooperate to form an image of an inside of an ear at an intermediate image plane. The plenoptic sensor includes a microimaging array positioned at the intermediate image plane and a sensor array positioned at a conjugate of the pupil plane. An optional filter module may be positioned at the pupil plane or one of its conjugates to facilitate three-dimensional and/or spectral imaging.

Abstract: The disclosure includes a system and method for performing image rectification using a single image and information identified from the single image. An image recognition application receives an input image, identifies a plurality of objects in the input image, estimates rectification parameters for the plurality of objects, identifies a plurality of candidate rectification parameters using a voting procedure on the rectification parameters for the plurality of objects, estimates final rectification parameters based on the plurality of candidate rectification parameters, computes a global transformation matrix using the final rectification parameters, and performs image rectification on the input image using the global transformation matrix.

Abstract: A plenoptic otoscope captures images used in making a medical diagnosis. For example, the plenoptic data can be processed to produce enhanced imagery of the ear interior, and this enhanced imagery can be used in making a medical diagnosis.

Abstract: A system and method for determining an object or product represented in an image is disclosed. The system receives a first image, determines a region of interest in the first image, determines a classification score for the region of interest using a convolutional neural network that assigns the region of interest the classification score corresponding to a class, and identifies a first product in the first image based on the classification score.

Abstract: A system and method for determining an object or product represented in an image is disclosed. The system receives a first image, determines a region of interest in the first image, determines a classification score for the region of interest using a convolutional neural network that assigns the region of interest the classification score corresponding to a class, and identifies a first product in the first image based on the classification score.

Abstract: A scene is segmented into objects based on light field data for the scene, including based on image pixel values (e.g., intensity) and disparity map(s). In one aspect, the light field data is used to estimate one or more disparity maps for the scene taken from different viewpoints. The scene is then segmented into a plurality of regions that correspond to objects in the scene. Unlike other approaches, the regions can be variable-depth. In one approach, the regions are defined by boundaries. The boundaries are determined by varying the boundary to optimize an objective function for the region defined by the boundary. The objective function is based in part on a similarity function that measures a similarity of image pixel values for pixels within the boundary and also measures a similarity of disparities for pixels within the boundary.

Abstract: A mobile information gateway comprises: a wearable human interface module having an image delivery and display mechanism for presenting information overlaid upon a wide field of view, a computing and communication module adapted to send and receive information to and from the human interface module; and a backend service server coupled for processing medical data. The present invention also includes a method comprising: capturing information with a first mobile information gateway device; processing the captured information to determine an identity of a first user and authenticate the first user; processing the captured information to determine an identity of a patient and authenticate the patient; processing the identity of the first user and the identity of the patient to retrieve information of the patient; and resenting the retrieved information of the patient overlaid on a field of view by the first mobile information gateway device.

Abstract: A mobile information gateway comprises: a wearable human interface module having an image delivery and display mechanism for presenting information overlaid upon a wide field of view, a computing and communication module adapted receive information from the human interface module and adapted to send commands and information to the human interface module including information for presentation; and a backend service server coupled for processing data from the computing and communication module including user identification and verification. The present invention also includes a method comprising capturing information with a wearable human interface module; processing the captured information to determine an identity of a first customer; processing the identity of the first customer to retrieve information related to the first customer; and presenting the information related to the first customer with the wearable human interface module overlaid upon a field of view.

Abstract: The disclosure includes a system and method for performing image rectification using a single image and information identified from the single image. An image recognition application receives an input image, identifies a plurality of objects in the input image, estimates rectification parameters for the plurality of objects, identifies a plurality of candidate rectification parameters using a voting procedure on the rectification parameters for the plurality of objects, estimates final rectification parameters based on the plurality of candidate rectification parameters, computes a global transformation matrix using the final rectification parameters, and performs image rectification on the input image using the global transformation matrix.

Abstract: A system and method that calculates a yaw error in an image and provides a user interface to a user for correcting the yaw error. The method includes receiving an image, performing line detection in the image, computing a line parameterization for lines in the image, computing a yaw angle for the image and providing the yaw data calculated in the image.