Abstract: A retina image template matching method is based on the registration and comparison between the images captured with portable low-cost fundus cameras (e.g., a consumer grade camera typically incorporated into a smartphone or tablet computer) and a baseline image. The method solves the challenges posed by registering small and low-quality retinal template images captured with such cameras. Our method combines dimension reduction methods with a mutual information (MI) based image registration technique. In particular, principle components analysis (PCA) and optionally block PCA are used as a dimension reduction method to localize the template image coarsely to the baseline image, then the resulting displacement parameters are used to initialize the MI metric optimization for registration of the template image with the closest region of the baseline image.

Abstract: An augmented reality (AR) device is described with a display system configured to adjust an apparent distance between a user of the AR device and virtual content presented by the AR device. The AR device includes a first tunable lens that change shape in order to affect the position of the virtual content. Distortion of real-world content on account of the changes made to the first tunable lens is prevented by a second tunable lens that changes shape to stay substantially complementary to the optical configuration of the first tunable lens. In this way, the virtual content can be positioned at almost any distance relative to the user without degrading the view of the outside world or adding extensive bulk to the AR device. The augmented reality device can also include tunable lenses for expanding a field of view of the augmented reality device.

Abstract: Techniques for tracking eye movement in an augmented reality system identify a plurality of base images of an object or a portion thereof. A search image may be generated based at least in part upon at least some of the plurality of base images. A deep learning result may be generated at least by performing a deep learning process on a base image using a neural network in a deep learning mode. A captured image may be localized at least by performing an image registration process on the captured image and the search image using a Kalman filter model and the deep learning result.

Abstract: A visual perception device has a look-up table stored in a laser driver chip. The look-up table includes relational gain data to compensate for brighter areas of a laser pattern wherein pixels are located more closely than areas where the pixels are further apart and to compensate for differences in intensity of individual pixels when the intensities of pixels are altered due to design characteristics of an eye piece.

Abstract: In a display system configured to display images at multiple depth planes, images perceived to be substantially full color or multi-colored may be formed using component color images that are distributed in unequal numbers across a plurality of depth planes. The distribution of component color images across the depth planes may vary based on color. In some embodiments, a display system includes a stack of waveguides that each output light of a particular color, with some colors having fewer numbers of associated waveguides than other colors. The stack of waveguides may include multiple pluralities of different numbers of waveguides, each plurality configured to produce an image by outputting light corresponding to a particular color. A display controller can introduce blur and/or change pixel size for images to be presented for at least one depth plane, based on the contents of images.

Abstract: A virtual image generation system comprises a planar optical waveguide having opposing first and second faces, an in-coupling (IC) element configured for optically coupling a collimated light beam from an image projection assembly into the planar optical waveguide as an in-coupled light beam, a first orthogonal pupil expansion (OPE) element associated with the first face of the planar optical waveguide for splitting the in-coupled light beam into a first set of orthogonal light beamlets, a second orthogonal pupil expansion (OPE) element associated with the second face of the planar optical waveguide for splitting the in-coupled light beam into a second set of orthogonal light beamlets, and an exit pupil expansion (EPE) element associated with the planar optical waveguide for splitting the first and second sets of orthogonal light beamlets into an array of out-coupled light beamlets that exit the planar optical waveguide.

Abstract: A display subsystem for a virtual image generation system for use by an end user comprises a planar waveguide apparatus, an optical fiber, at least one light source configured for emitting light from a distal end of the optical fiber, and a collimation element mounted to a distal end of the optical fiber for collimating light from the optical fiber. The virtual image generation system further comprises a mechanical drive assembly to which the optical fiber is mounted to the drive assembly. The mechanical drive assembly is configured for displacing the distal end of the optical fiber, along with the collimation element, in accordance with a scan pattern. The virtual image generation system further comprises an optical waveguide input apparatus configured for directing the collimated light from the collimation element down the planar waveguide apparatus, such that the planar waveguide apparatus displays image frames to the end user.

Abstract: An eye tracking system includes a pair of glasses including two frames, a projector coupled to the pair of glasses and operable to project a beam of light, and an eyepiece mounted in one of the two frames and optically coupled to the projector. The eyepiece is operable to direct at least a portion of the beam of light towards an eye of a user. The eye tracking system also includes one or more optical sensors coupled to at least one of the two frames of the pair of glasses and operable to detect a set of reflected signals off the eye of the user, and a processor coupled to the projector and the one or more optical sensors, wherein the processor is operable to determine an eye orientation based on the set of reflected signals.

Abstract: A virtual image generation system comprises a planar optical waveguide having opposing first and second faces, an in-coupling (IC) element configured for optically coupling a collimated light beam from an image projection assembly into the planar optical waveguide as an in-coupled light beam, a first orthogonal pupil expansion (OPE) element associated with the first face of the planar optical waveguide for splitting the in-coupled light beam into a first set of orthogonal light beamlets, a second orthogonal pupil expansion (OPE) element associated with the second face of the planar optical waveguide for splitting the in-coupled light beam into a second set of orthogonal light beamlets, and an exit pupil expansion (EPE) element associated with the planar optical waveguide for splitting the first and second sets of orthogonal light beamlets into an array of out-coupled light beamlets that exit the planar optical waveguide.

Abstract: Augmented reality and virtual reality display systems and devices are configured for efficient use of projected light. In some aspects, a display system includes a light projection system and a head-mounted display configured to project light into an eye of the user to display virtual image content. The head-mounted display includes at least one waveguide comprising a plurality of in-coupling regions each configured to receive, from the light projection system, light corresponding to a portion of the user's field of view and to in-couple the light into the waveguide; and a plurality of out-coupling regions configured to out-couple the light out of the waveguide to display the virtual content, wherein each of the out-coupling regions are configured to receive light from different ones of the in-coupling regions. In some implementations, each in-coupling region has a one-to-one correspondence with a unique corresponding out-coupling region.

Abstract: One embodiment is directed to a user display device comprising a housing frame mountable on the head of the user, a lens mountable on the housing frame and a projection sub system coupled to the housing frame to determine a location of appearance of a display object in a field of view of the user based at least in part on at least one of a detection of a head movement of the user and a prediction of a head movement of the user, and to project the display object to the user based on the determined location of appearance of the display object.

Abstract: A semiconductor substrate includes a first semiconductor layer, a first dielectric layer coupled to the first semiconductor layer, and a second semiconductor layer coupled to the first dielectric layer. The second semiconductor layer includes a base portion substantially aligned with the first dielectric layer and a cantilever portion protruding from an end of the first dielectric layer. The cantilever portion includes a tapered surface tapering from a bottom surface of the second semiconductor layer toward a top surface of the second semiconductor layer.

Abstract: Techniques for tracking eye movement in an augmented reality system identify a plurality of base images of an object or a portion thereof. A search image may be generated based at least in part upon at least some of the plurality of base images. A deep learning result may be generated at least by performing a deep learning process on a base image using a neural network in a deep learning mode. A captured image may be localized at least by performing an image registration process on the captured image and the search image using a Kalman filter model and the deep learning result.

Abstract: A display subsystem for a virtual image generation system for use by an end user comprises a planar waveguide apparatus, an optical fiber, at least one light source configured for emitting light from a distal end of the optical fiber, and a collimation element mounted to a distal end of the optical fiber for collimating light from the optical fiber. The virtual image generation system further comprises a mechanical drive assembly to which the optical fiber is mounted to the drive assembly. The mechanical drive assembly is configured for displacing the distal end of the optical fiber, along with the collimation element, in accordance with a scan pattern. The virtual image generation system further comprises an optical waveguide input apparatus configured for directing the collimated light from the collimation element down the planar waveguide apparatus, such that the planar waveguide apparatus displays image frames to the end user.

Abstract: An augmented reality display system is configured to direct a plurality of parallactically-disparate intra-pupil images into a viewer's eye. The parallactically-disparate intra-pupil images provide different parallax views of a virtual object, and impinge on the pupil from different angles. In the aggregate, the wavefronts of light forming the images approximate a continuous divergent wavefront and provide selectable accommodation cues for the user, depending on the amount of parallax disparity between the intra-pupil images. The amount of parallax disparity is selected using a light source that outputs light for different images from different locations, with spatial differences in the locations of the light output providing differences in the paths that the light takes to the eye, which in turn provide different amounts of parallax disparity.

Abstract: Augmented reality and virtual reality display systems and devices are configured for efficient use of projected light. In some aspects, a display system includes a light projection system and a head-mounted display configured to project light into an eye of the user to display virtual image content. The head-mounted display includes at least one waveguide comprising a plurality of in-coupling regions each configured to receive, from the light projection system, light corresponding to a portion of the user's field of view and to in-couple the light into the waveguide; and a plurality of out-coupling regions configured to out-couple the light out of the waveguide to display the virtual content, wherein each of the out-coupling regions are configured to receive light from different ones of the in-coupling regions. In some implementations, each in-coupling region has a one-to-one correspondence with a unique corresponding out-coupling region.

Abstract: One embodiment is directed to a user display device comprising a housing frame mountable on the head of the user, a lens mountable on the housing frame and a projection sub system coupled to the housing frame to determine a location of appearance of a display object in a field of view of the user based at least in part on at least one of a detection of a head movement of the user and a prediction of a head movement of the user, and to project the display object to the user based on the determined location of appearance of the display object.

Abstract: Architectures are provided for selectively outputting light for forming images, the light having different wavelengths and being outputted with low levels of crosstalk. In some embodiments, light is incoupled into a waveguide and deflected to propagate in different directions, depending on wavelength. The incoupled light then outcoupled by outcoupling optical elements that outcouple light based on the direction of propagation of the light. In some other embodiments, color filters are between a waveguide and outcoupling elements. The color filters limit the wavelengths of light that interact with and are outcoupled by the outcoupling elements. In yet other embodiments, a different waveguide is provided for each range of wavelengths to be outputted. Incoupling optical elements selectively incouple light of the appropriate range of wavelengths into a corresponding waveguide, from which the light is outcoupled.

Abstract: A method of operating an eyepiece waveguide includes directing light from a projector to impinge on an incoupling grating (ICG). The method also includes diffracting a first fraction of the light from the projector into a first portion of the eyepiece waveguide, propagating the first fraction of the light into a second portion of the eyepiece waveguide, and diffracting the first fraction of the light out of the eyepiece waveguide. The method further includes diffracting a second fraction of the light from the projector into the second portion of the eyepiece waveguide, propagating the second fraction of the light into the first portion of the eyepiece waveguide, and diffracting the second fraction out of the eyepiece waveguide.

Abstract: An augmented reality display system is configured to direct a plurality of parallactically-disparate intra-pupil images into a viewer's eye. The parallactically-disparate intra-pupil images provide different parallax views of a virtual object, and impinge on the pupil from different angles. In the aggregate, the wavefronts of light forming the images approximate a continuous divergent wavefront and provide selectable accommodation cues for the user, depending on the amount of parallax disparity between the intra-pupil images. The amount of parallax disparity is selected using a light source that outputs light for different images from different locations, with spatial differences in the locations of the light output providing differences in the paths that the light takes to the eye, which in turn provide different amounts of parallax disparity.