Abstract: An exit pupil expander (EPE) has entrance and exit pupils, a back surface adjacent to the entrance pupil, and an opposed front surface. In one embodiment the EPE is geometrically configured such that light defining a center wavelength that enters at the entrance pupil perpendicular to the back surface experiences angularly varying total internal reflection between the front and back surfaces such that the light exiting the optical channel perpendicular to the exit pupil is at a wavelength shifted from the center wavelength. In another embodiment a first distance at the entrance pupil between the front and back surfaces is different from a second distance at the exit pupil between the front and back surfaces. The EPE may be deployed in a head-wearable imaging device (e.g., virtual or augmented reality) where the entrance pupil in-couples light from a micro display and the exit pupil out-couples light from the EPE.

Abstract: A wearable device for projecting image light to an eye of a viewer and forming an image of virtual content in an augmented reality display is provided. The wearable device includes a projector and stack of waveguides optically connected to the projector. The wearable device also includes an eye tracking system comprising a plurality of illumination sources, an optical element having optical power, and a set of cameras. The optical element is disposed between the plurality of illumination sources and the set of cameras. In some embodiments, the augmented reality display includes an eyepiece operable to output virtual content from an output region and a plurality of illumination sources. At least some of the plurality of illumination sources overlap with the output region.

Abstract: An audio system and method of spatially rendering audio signals that uses modified virtual speaker panning is disclosed. The audio system may include a fixed number F of virtual speakers, and the modified virtual speaker panning may dynamically select and use a subset P of the fixed virtual speakers. The subset P of virtual speakers may be selected using a low energy speaker detection and culling method, a source geometry-based culling method, or both. One or more processing blocks in the decoder/virtualizer may be bypassed based on the energy level of the associated audio signal or the location of the sound source relative to the user/listener, respectively. In some embodiments, a virtual speaker that is designated as an active virtual speaker at a first time, may also be designated as an active virtual speaker at a second time to ensure the processing completes.

Abstract: A method for displaying virtual content to a user, the method includes determining an accommodation of the user's eyes. The method also includes delivering, through a first waveguide of a stack of waveguides, light rays having a first wavefront curvature based at least in part on the determined accommodation, wherein the first wavefront curvature corresponds to a focal distance of the determined accommodation. The method further includes delivering, through a second waveguide of the stack of waveguides, light rays having a second wavefront curvature, the second wavefront curvature associated with a predetermined margin of the focal distance of the determined accommodation.

Abstract: A method of displaying a reconstructed foveated image includes receiving an image having a first resolution, determining an eye gaze location and velocity. The method also includes processing N-1 sections of N sections to produce N-1 first quality sections based on the eye gaze location and velocity, processing one section of the N sections to provide one second quality section based on the eye gaze location and velocity, and combining the N-1 first quality sections and the one second quality section to form a foveated image having a plurality of lines of pixel data, each line being defined by a plurality of pixel groups. The method further includes forming a mask for each line, providing pixel values for pixels in pixel groups having the second bit, forming a reconstructed foveated image using the mask and the provided pixel values, and transmitting the reconstructed foveated image to a display.

Abstract: Several unique configurations for interferometric recording of volumetric phase diffractive elements with relatively high angle diffraction for use in waveguides are disclosed. Separate layer EPE and OPE structures produced by various methods may be integrated in side-by-side or overlaid constructs, and multiple such EPE and OPE structures may be combined or multiplexed to exhibit EPE/OPE functionality in a single, spatially-coincident layer. Multiplexed structures reduce the total number of layers of materials within a stack of eyepiece optics, each of which may be responsible for displaying a given focal depth range of a volumetric image.

Abstract: Methods, systems, and wearable extended reality devices for generating a floorplan of an indoor scene are provided. A room classification of a room and a wall classification of a wall for the room may be determined from an input image of the indoor scene. A floorplan may be determined based at least in part upon the room classification and the wall classification without constraining a total number of rooms in the indoor scene or a size of the room.

Abstract: Eyepieces and methods of fabricating the eyepieces are disclosed. In some embodiments, the eyepiece comprises a curved cover layer and a waveguide layer for propagating light. In some embodiments, the curved cover layer comprises an antireflective feature.

Abstract: This disclosure is related to systems and methods for rendering audio for a mixed reality environment. Methods according to embodiments of this disclosure include receiving an input audio signal, via a wearable device in communication with a mixed reality environment, the input audio signal corresponding to a sound source originating from a real environment. In some embodiments, the system can determine one or more acoustic properties associated with the mixed reality environment. In some embodiments, the system can determine a signal modification parameter based on the one or more acoustic properties associated with the mixed reality environment. In some embodiments, the system can apply the signal modification parameter to the input audio signal to determine a second audio signal. The system can present the second audio signal to the user.

Abstract: An eyewear device for being worn on a head of a user for presenting virtual content to a user comprises a frame structure having a frame front, and an optical assembly having a first rigidity. The optical assembly has a chassis and a plurality of optical components affixed to the chassis. The eyewear device further comprises a plurality of mounts mechanically coupling the chassis of the optical assembly to the frame front, at least one of the plurality of mounts having a second rigidity less than the first rigidity, such that the mount(s) is configured for preventing at least a portion of a first static deformation load applied to the frame front from being mechanically communicated to the optical assembly.

Abstract: A waveguide apparatus includes a planar waveguide and at least one optical diffraction element (DOE) that provides a plurality of optical paths between an exterior and interior of the planar waveguide. A phase profile of the DOE may combine a linear diffraction grating with a circular lens, to shape a wave front and produce beams with desired focus. Waveguide apparati may be assembled to create multiple focal planes. The DOE may have a low diffraction efficiency, and planar waveguides may be transparent when viewed normally, allowing passage of light from an ambient environment (e.g., real world) useful in AR systems. Light may be returned for temporally sequentially passes through the planar waveguide. The DOE(s) may be fixed or may have dynamically adjustable characteristics. An optical coupler system may couple images to the waveguide apparatus from a projector, for instance a biaxially scanning cantilevered optical fiber tip.

Abstract: Techniques for operating an optical system are disclosed. World light may be linearly polarized along a first axis. When the optical system is operating in accordance with a first state, a polarization of the world light may be rotated by 90 degrees, the world light may be linearly polarized along a second axis perpendicular to the first axis, and zero net optical power may be applied to the world light. When the optical system is operating in accordance with a second state, virtual image light may be projected onto an eyepiece of the optical system, the world light and the virtual image light may be linearly polarized along the second axis, a polarization of the virtual image light may be rotated by 90 degrees, and non-zero net optical power may be applied to the virtual image light.

Abstract: Embodiments of this disclosure provides systems and methods for displays. In embodiments, a display system includes a frame, an eyepiece coupled to the frame, and a first adhesive bond disposed between the frame and the eyepiece. The eyepiece can include a light input region and a light output region. The first adhesive bond can be disposed along a first portion of a perimeter of the eyepiece, where the first portion of the perimeter of the eyepiece borders the light input region such that the first adhesive bond is configured to maintain a position of the light input region relative to the frame.

Abstract: Systems and methods for estimating a layout of a room are disclosed. The room layout can comprise the location of a floor, one or more walls, and a ceiling. In one aspect, a neural network can analyze an image of a portion of a room to determine the room layout. The neural network can comprise a convolutional neural network having an encoder sub-network, a decoder sub-network, and a side sub-network. The neural network can determine a three-dimensional room layout using two-dimensional ordered keypoints associated with a room type. The room layout can be used in applications such as augmented or mixed reality, robotics, autonomous indoor navigation, etc.

Abstract: An exit pupil expander (EPE) has entrance and exit pupils, a back surface adjacent to the entrance pupil, and an opposed front surface. In one embodiment the EPE is geometrically configured such that light defining a center wavelength that enters at the entrance pupil perpendicular to the back surface experiences angularly varying total internal reflection between the front and back surfaces such that the light exiting the optical channel perpendicular to the exit pupil is at a wavelength shifted from the center wavelength. In another embodiment a first distance at the entrance pupil between the front and back surfaces is different from a second distance at the exit pupil between the front and back surfaces. The EPE may be deployed in a head-wearable imaging device (e.g., virtual or augmented reality) where the entrance pupil in-couples light from a micro display and the exit pupil out-couples light from the EPE.

Abstract: Apparatus and methods for displaying an image by a rotating structure are provided. The rotating structure can comprise blades of a fan. The fan can be a cooling fan for an electronics device such as an augmented reality display. In some embodiments, the rotating structure comprises light sources that emit light to generate the image. The light sources can comprise light-field emitters. In other embodiments, the rotating structure is illuminated by an external (e.g., non-rotating) light source.

Abstract: A method of viewing a patient including inserting a catheter is described for health procedure navigation. A CT scan is carried out on a body part of a patient. Raw data from the CT scan is processed to create three-dimensional image data, storing the image data in the data store. Projectors receive generated light in a pattern representative of the image data and waveguides guide the light to a retina of an eye of a viewer while light from an external surface of the body transmits to the retina of the eye so that the viewer sees the external surface of the body augmented with the processed data rendering of the body part.

Abstract: Techniques are disclosed for operating a time-of-flight (TOF) sensor. The TOF may be operated in a low power mode by repeatedly performing a low power mode sequence, which may include performing a depth frame by emitting light pulses, detecting reflected light pulses, and computing a depth map based on the detected reflected light pulses. Performing the low power mode sequence may also include performing an amplitude frame at least one time by emitting a light pulse, detecting a reflected light pulse, and computing an amplitude map based on the detected reflected light pulse. In response to determining that an activation condition is satisfied, the TOF may be switched to operate in a high accuracy mode by repeatedly performing a high accuracy mode sequence, which may include performing the depth frame multiple times.

Abstract: An optical system comprises an optically transmissive substrate comprising a multilevel metasurface which comprises a grating comprising a plurality of multilevel unit cells. Each unit cell comprises, on a lowermost level, a laterally-elongated first lowermost level nanobeam having a first width and a laterally-elongated second lowermost level nanobeam having a second width larger than the first width. Each unit cell further comprises, on an uppermost level, a laterally-elongated first uppermost level nanobeam above the first lowermost level nanobeam and a laterally-elongated second uppermost level nanobeam above the second lowermost level nanobeam.

Abstract: Apparatuses and methods for displaying a 3-D representation of an object are described. Apparatuses can include a rotatable structure, motor, and multiple light field sub-displays disposed on the rotatable structure. The apparatuses can store a light field image to be displayed, the light field image providing multiple different views of the object at different viewing directions. A processor can drive the motor to rotate the rotatable structure and map the light field image to each of the light field sub-displays based in part on the rotation angle, and illuminate the light field sub-displays based in part on the mapped light field image. The apparatuses can include a display panel configured to be viewed from a fiducial viewing direction, where the display panel is curved out of a plane that is perpendicular to the fiducial viewing direction, and a plurality of light field sub-displays disposed on the display panel.