Abstract: Systems and methods for eye pose identification using features of an eye are described. Embodiments of the systems and methods can include segmenting an iris of an eye in the eye image to obtain pupillary and limbic boundaries of the eye, determining two angular coordinates (e.g., pitch and yaw) of an eye pose using the pupillary and limbic boundaries of the eye, identifying an eye feature of the eye (e.g., an iris feature or a scleral feature), determining a third angular coordinate (e.g., roll) of the eye pose using the identified eye feature, and utilizing the eye pose measurement for display of an image or a biometric application. In some implementations, iris segmentation may not be performed, and the two angular coordinates are determined from eye features.

Abstract: An apparatus is disclosed for capturing image information. The apparatus includes a waveguide having opposed planar input and output faces. A diffractive optical element (DOE) is formed across the waveguide. The DOE is configured to couple a portion of the light passing through the waveguide into the waveguide. The light coupled into the waveguide is directed via total internal reflection to an exit location on the waveguide. The apparatus further includes a light sensor having an input positioned adjacent the exit location of the waveguide to capture light exiting therefrom and generate output signals corresponding thereto. A processor determines the angle and position of the coupled light with respect to the input face of the waveguide based on the output signals.

Abstract: Systems and methods for eyelid shape estimation are disclosed. In one aspect, after receiving an eye image of an eye (e.g., from an image capture device), an eye pose of the eye in the eye image is determined. From the eye pose, an eyelid shape (of an upper eyelid or a lower eyelid) can be estimated using an eyelid shape mapping model. The eyelid shape mapping model relates the eye pose and the eyelid shape. In another aspect, the eyelid shape mapping model is learned (e.g., using a neural network).

Abstract: Disclosed herein are systems and methods for device authentication or pairing. In an aspect, a wearable display system comprises a display, an image capture device configured to capture images of a companion device, a computer-readable storage medium configured to store the images of the companion device, and a processor in communication with the image capture device and the storage medium. The processor can be programmed with executable instructions to receive a first image of a first optical pattern displayed by the companion device captured by the image capture device, wherein the first optical pattern is generated by the companion device based on first shared data, extract first data from the first optical pattern in the received first image, authenticate the companion device based on the first data extracted from the first optical pattern, and notify a user of the wearable display system that the companion device is authenticated.

Abstract: Systems and methods for blue light adjustment with a wearable display system are provided. Embodiments of the systems and methods for blue light adjustment can include receiving an eye image of an eye exposed to an adjusted level of blue light; detecting a change in a pupillary response by comparison of the received eye image to a first image; determining that the pupillary response corresponds to a biometric characteristic of a human individual; and allowing access to a biometric application based on the pupillary response determination.

Abstract: Systems and methods for blue light adjustment with a wearable display system are provided. Embodiments of the systems and methods for blue light adjustment can include receiving an initial eye image obtained by an image capture device; adjusting a level of blue light exposed to an eye associated with the initial eye image; receiving an adjustment eye image of the eye exposed to the adjusted level of blue light; detecting a change in a pupillary response of the adjustment eye image relative to the initial eye image; determining that the detected change in the pupillary response passes a biometric application threshold; and utilizing eye images or the detected change in the pupillary response for a biometric application.

Abstract: Systems and methods for eyelid shape estimation are disclosed. In one aspect, after receiving an eye image of an eye (e.g., from an image capture device), an eye-box is generated over an iris of the eye in the eye image. A plurality of radial lines can be generated from approximately the center of the eye-box to an upper edge or a lower edge of the eye box. Candidate points can be determined to have local maximum derivatives along the plurality of radial lines. From the candidate points, an eyelid shape curve (e.g., for an upper eyelid or a lower eyelid) can be determined by fitting a curve (e.g., a parabola or a polynomial) to the candidate points or a subset of the candidate points.

Abstract: Systems and methods pertaining to iris recognition are disclosed. A system is disclosed comprising an iris imager configured to acquire an iris image, and a processor configured to receive a first plurality of iris images from the iris imager, generate a first plurality of iris codes corresponding to the first plurality of iris images, generate a distribution metric corresponding to an iris cell location, generate a first composite iris code using the distribution metric, and generate a first match value using the first composite iris code and a first stored iris code.

Abstract: An apparatus is disclosed for capturing image information. The apparatus includes a waveguide having opposed planar input and output faces. A diffractive optical element (DOE) is formed across the waveguide. The DOE is configured to couple a portion of the light passing through the waveguide into the waveguide. The light coupled into the waveguide is directed via total internal reflection to an exit location on the waveguide. The apparatus further includes a light sensor having an input positioned adjacent the exit location of the waveguide to capture light exiting therefrom and generate output signals corresponding thereto. A processor determines the angle and position of the coupled light with respect to the input face of the waveguide based on the output signals.

Abstract: An area monitoring system includes a camera to monitor a region of interest and provide real-time video data corresponding to real-time video images of the region of interest. A memory stores data associated with a rendered virtual environment associated with a geographic region that includes the region of interest. An image processor combines the real-time video data and the data associated with the virtual environment to generate image data. A user interface enables a user to navigate the virtual environment via inputs and displays the image data based on a location perspective corresponding to a viewing perspective of the user at a given virtual location within the virtual environment. The image processor superimposes the video images of the region of interest onto the virtual environment in the image data at a location and an orientation associated with the location perspective of the user within the virtual environment.

Abstract: This disclosure is directed to an image sensor. The image sensor includes a two-dimensional pixel array divided into a plurality of blocks, each of the plurality of blocks comprising pixels arranged in at least two different rows and two different columns, and a shutter mechanism that exposes the plurality of blocks sequentially, with all pixels in each block being exposed synchronously.

Abstract: A wearable display system can automatically recognize a physical remote or a device that the remote serves using computer vision techniques. The wearable system can generate a virtual remote with a virtual control panel viewable and interactable by user of the wearable system. The virtual remote can emulate the functionality of the physical remote. The user can select a virtual remote for interaction, for example, by looking or pointing at the parent device or its remote control, or by selecting from a menu of known devices. The virtual remote may include a virtual button, which is associated with a volume in the physical space. The wearable system can detect that a virtual button is actuated by determining whether a portion of the user's body (e.g., the user's finger) has penetrated the volume associated with the virtual button.

Abstract: An apparatus is disclosed for capturing image information. The apparatus includes a waveguide having opposed planar input and output faces. A diffractive optical element (DOE) is formed across the waveguide. The DOE is configured to couple a portion of the light passing through the waveguide into the waveguide. The light coupled into the waveguide is directed via total internal reflection to an exit location on the waveguide. The apparatus further includes a light sensor having an input positioned adjacent the exit location of the waveguide to capture light exiting therefrom and generate output signals corresponding thereto. A processor determines the angle and position of the coupled light with respect to the input face of the waveguide based on the output signals.

Abstract: Disclosed are improved methods, systems and devices for color night vision that reduce the number of intensifiers and/or decrease noise. In some embodiments, color night vision is provided in system in which multiple spectral bands are maintained, filtered separately, and then recombined in a unique three-lens-filtering setup. An illustrative four-camera night vision system is unique in that its first three cameras separately filter different bands using a subtractive Cyan, Magenta and Yellow (CMY) color filtering-process, while its fourth camera is used to sense either additional IR illuminators or a luminance channel to increase brightness. In some embodiments, the color night vision is implemented to distinguish details of an image in low light. The unique application of the three-lens subtractive CMY filtering allows for better photon scavenging and preservation of important color information.

Abstract: Systems and methods for synthesizing an image of the face by a head-mounted device (HMD) are disclosed. The HMD may not be able to observe a portion of the face. The systems and methods described herein can generate a mapping from a conformation of the portion of the face that is not imaged to a conformation of the portion of the face observed. The HMD can receive an image of a portion of the face and use the mapping to determine a conformation of the portion of the face that is not observed. The HMD can combine the observed and unobserved portions to synthesize a full face image.

Abstract: A plan through a space having a near field and a far field is determined. Using a sensor device, measurements of the far field are obtained and stored in an electronic memory. A processor uses the measurements to determine the viability of each far field plan among a plurality of candidate far field plans. The processor also determines a flexibility score for each of the candidate far field plans and selects a composite plan comprising the viable far field plan having a highest flexibility score among the viable candidate far field plans.

Abstract: Systems and methods are provided for extracting application relevant data from messages. In one embodiment, a system can comprise a message parser that parses messages and builds a message tree having one or more objects, one or more data type templates that define a given data type based on one or more data elements and a comparison engine that matches data elements in the one or more objects with data elements in the one or more data type templates. The comparison engine groups data elements in the one or more objects that matches data elements in the one or more data templates as a specific data type corresponding to the associated data type template that is matched.

Abstract: This disclosure is directed to an image sensor. The image sensor includes a two-dimensional pixel array divided into a plurality of blocks, each of the plurality of blocks comprising pixels arranged in at least two different rows and two different columns, and a shutter mechanism that exposes the plurality of blocks sequentially, with all pixels in each block being exposed synchronously.

Abstract: A wearable device can include an inward-facing imaging system configured to acquire images of a user's periocular region. The wearable device can determine a relative position between the wearable device and the user's face based on the images acquired by the inward-facing imaging system. The relative position may be used to determine whether the user is wearing the wearable device, whether the wearable device fits the user, or whether an adjustment to a rendering location of virtual object should be made to compensate for a deviation of the wearable device from its normal resting position.

Abstract: Systems and methods for eye image segmentation and image quality estimation are disclosed. In one aspect, after receiving an eye image, a device such as an augmented reality device can process the eye image using a convolutional neural network with a merged architecture to generate both a segmented eye image and a quality estimation of the eye image. The segmented eye image can include a background region, a sclera region, an iris region, or a pupil region. In another aspect, a convolutional neural network with a merged architecture can be trained for eye image segmentation and image quality estimation. In yet another aspect, the device can use the segmented eye image to determine eye contours such as a pupil contour and an iris contour. The device can use the eye contours to create a polar image of the iris region for computing an iris code or biometric authentication.