Abstract: Techniques are disclosed for performing localization of a handheld device with respect to a wearable device. At least one sensor mounted to the handheld device, such as an inertial measurement unit (IMU), may obtain handheld data indicative of movement of the handheld device with respect to the world. An imaging device mounted to either the handheld device or the wearable device may capture a fiducial image containing a number of fiducials affixed to the other device. The number of fiducials contained in the image are determined. Based on the number of fiducials, at least one of a position and an orientation of the handheld device with respect to the wearable device are updated based on the image and the handheld data in accordance with a first operating state, a second operating state, or a third operating state.

Abstract: An eyepiece for an augmented reality display system. The eyepiece can include a waveguide substrate. The waveguide substrate can include an input coupler grating (ICG), an orthogonal pupil expander (OPE) grating, a spreader grating, and an exit pupil expander (EPE) grating. The ICG can couple at least one input light beam into at least a first guided light beam that propagates inside the waveguide substrate. The OPE grating can divide the first guided light beam into a plurality of parallel, spaced-apart light beams. The spreader grating can receive the light beams from the OPE grating and spread their distribution. The spreader grating can include diffractive features oriented at approximately 90° to diffractive features of the OPE grating. The EPE grating can re-direct the light beams from the first OPE grating and the first spreader grating such that they exit the waveguide substrate.

Abstract: An example head-mounted display device includes a light projector and an eyepiece. The eyepiece includes a light guiding layer and a first focusing optical element. The first focusing optical element includes a first region having a first optical power, and a second region having a second optical power different from the first optical power. The light guiding layer is configured to: i) receive light from the light projector, ii) direct at least a first portion of the light to a user's eye through the first region to present a first virtual image to the user at a first focal distance, and iii) direct at least a second portion of the light to the user's eye through the second region to present a second virtual image to the user at a second focal distance.

Abstract: Disclosed herein are systems and methods for distributed computing and/or networking for mixed reality systems. A method may include capturing an image via a camera of a head-wearable device. Inertial data may be captured via an inertial measurement unit of the head-wearable device. A position of the head-wearable device can be estimated based on the image and the inertial data via one or more processors of the head-wearable device. The image can be transmitted to a remote server. A neural network can be trained based on the image via the remote server. A trained neural network can be transmitted to the head-wearable device.

Abstract: An eyepiece waveguide for an augmented reality. The eyepiece waveguide can include a transparent substrate with an input coupler region, a first orthogonal pupil expander (OPE) region, and an exit pupil expander (EPE) region. The input coupler region can couple an input light beam that is externally incident on the input coupler region into at least a first guided light beam that propagates inside the substrate. The first OPE region can divide the first guided beam into a plurality of replicated, spaced-apart beams. The EPE region can re-direct the replicated beams from the first OPE region such that they exit the substrate. The EPE region can have an amount of optical power.

Abstract: An eyepiece for a head-mounted display includes one or more first waveguides arranged to receive light from a spatial light modulator at a first edge, guide at least some of the received light to a second edge opposite the first edge, and extract at least some of the light through a face of the one or more first waveguides between the first and second edges. The eyepiece also includes a second waveguide positioned to receive light exiting the one or more first waveguides at the second edge and guide the received light to one or more light absorbers.

Abstract: Techniques are disclosed for performing localization of a handheld device with respect to a wearable device. At least one sensor mounted to the handheld device, such as an inertial measurement unit (IMU), may obtain handheld data indicative of movement of the handheld device with respect to the world. An imaging device mounted to either the handheld device or the wearable device may capture a fiducial image containing a number of fiducials affixed to the other device. The number of fiducials contained in the image are determined. Based on the number of fiducials, at least one of a position and an orientation of the handheld device with respect to the wearable device are updated based on the image and the handheld data in accordance with a first operating state, a second operating state, or a third operating state.

Abstract: An eyepiece for a head-mounted display includes one or more first waveguides arranged to receive light from a spatial light modulator at a first edge, guide at least some of the received light to a second edge opposite the first edge, and extract at least some of the light through a face of the one or more first waveguides between the first and second edges. The eyepiece also includes a second waveguide positioned to receive light exiting the one or more first waveguides at the second edge and guide the received light to one or more light absorbers.

Abstract: An eyepiece for projecting an image includes a first planar waveguide positioned in a first lateral plane. The first planar waveguide comprises a first diffractive optical element (DOE) disposed at a first lateral position. The eyepiece also includes a second planar waveguide positioned in a second lateral plane adjacent to the first lateral plane. The second planar waveguide comprises a second DOE disposed at a second lateral position different from the first lateral position. The eyepiece further includes a third planar waveguide positioned in a third lateral plane adjacent to the second lateral plane. The third planar waveguide comprises a third DOE disposed at a third lateral position different from the first lateral position and the second lateral position. The eyepiece also includes an optical filter positioned between the second planar waveguide and the third planar waveguide. The optical filter is disposed at the third lateral position.

Abstract: A method for characterizing a digital color camera includes, for each of three primary colors used in a field sequential color virtual image, determining a conversion model for each color using RGB values and the color-measurement values. For each primary color, the method includes illuminating a display device using an input light beam of a primary color having spectral properties representative of a light beam in a virtual image in a wearable device. The method includes capturing, with the digital color camera, an image of the display device, and determining, from the image, RGB values for each primary color. The method includes capturing, with a color-measurement device, a color-measurement value associated with each corresponding primary color at the display device, thereby acquiring a color-measurement value in an absolute color space. A conversion model for each color is determined using RGB values and the color-measurement values.

Abstract: Techniques are described for using confinement structures and/or pattern gratings to reduce or prevent the wicking of sealant polymer (e.g., glue) into the optically active areas of a multi-layered optical assembly. A multi-layered optical structure may include multiple layers of substrate imprinted with waveguide grating patterns. The multiple layers may be secured using an edge adhesive, such as a resin, epoxy, glue, and so forth. A confinement structure such as an edge pattern may be imprinted along the edge of each layer to control and confine the capillary flow of the edge adhesive and prevent the edge adhesive from wicking into the functional waveguide grating patterns of the layers. Moreover, the edge adhesive may be carbon doped or otherwise blackened to reduce the reflection of light off the edge back into the interior of the layer, thus improving the optical function of the assembly.

Abstract: An eyepiece unit with optical filters includes a set of waveguide layers including a first waveguide layer and a second waveguide layer. The first waveguide layer is disposed in a first lateral plane and includes a first incoupling diffractive element disposed at a first lateral position, a first waveguide, and a first outcoupling diffractive element. The second waveguide layer is disposed in a second lateral plane adjacent to the first lateral plane and includes a second incoupling diffractive element disposed at a second lateral position, a second waveguide, and a second outcoupling diffractive element. The eyepiece unit also includes a set of optical filters including a first optical filter positioned at the first lateral position and operable to attenuate light outside a first spectral band and a second optical filter positioned at the second lateral position and operable to attenuate light outside a second spectral band.

Abstract: Techniques are disclosed for performing localization of a handheld device with respect to a wearable device. At least one sensor mounted to the handheld device, such as an inertial measurement unit (IMU), may obtain handheld data indicative of movement of the handheld device with respect to the world. An imaging device mounted to either the handheld device or the wearable device may capture a fiducial image containing a number of fiducials affixed to the other device. The number of fiducials contained in the image are determined. Based on the number of fiducials, at least one of a position and an orientation of the handheld device with respect to the wearable device are updated based on the image and the handheld data in accordance with a first operating state, a second operating state, or a third operating state.

Abstract: Techniques are described for inspecting optical devices, such as eyepieces, to determine whether they exhibit light leakage through an edge sealant that has been applied to the device. Embodiments provide an inspection apparatus that can be employed to detect the leakage of light through an edge sealant of an optical device, where the edge sealant is applied to prevent, or at least reduce, the leakage of light from the optical device. Light from a light source is projected into the optical device. The light can travel along one or more wave guides within the device, until reaching an edge of the device. Light that is able to leak through an edge sealant can be reflected, using mirror(s) in the apparatus, and detected by a camera. Image(s) captured by the camera can be analyzed to determine the performance of the optical device with respect to edge leakage.

Abstract: An optical device includes a waveguide including an in-coupling optical element configured to in-couple light into the waveguide, a light distributing element configured to receive light from the in-coupling optical element and distribute light at a selected wavelength, and an out-coupling optical element configured to receive light from the light distributing element and out-couple light out of the waveguide. The out-coupling optical element includes a first region configured to out-couple light at a first depth plane based on a lens function of the first region and a second region configured to out-couple light at a second depth plane based on a different lens function of the second region.

Abstract: Techniques are disclosed for performing localization of a handheld device with respect to a wearable device. At least one sensor mounted to the handheld device, such as an inertial measurement unit (IMU), may obtain handheld data indicative of movement of the handheld device with respect to the world. An imaging device mounted to either the handheld device or the wearable device may capture a fiducial image containing a number of fiducials affixed to the other device. The number of fiducials contained in the image are determined. Based on the number of fiducials, at least one of a position and an orientation of the handheld device with respect to the wearable device are updated based on the image and the handheld data in accordance with a first operating state, a second operating state, or a third operating state.

Abstract: A sensory eyewear system for a mixed reality device can facilitate user's interactions with the other people or with the environment. As one example, the sensory eyewear system can recognize and interpret a sign language, and present the translated information to a user of the mixed reality device. The wearable system can also recognize text in the user's environment, modify the text (e.g., by changing the content or display characteristics of the text), and render the modified text to occlude the original text.

Abstract: The invention provides a content provisioning system. A mobile device has a mobile device processor. The mobile device mobile device has communication interface connected to the mobile device processor and a first resource device communication interface and under the control of the mobile device processor to receive first content transmitted by the first resource device transmitter The mobile device mobile device has a mobile device output device connected to the mobile device processor and under control of the mobile device processor capable of providing an output that can be sensed by a user.

Abstract: A sensory eyewear system for a mixed reality device can facilitate user's interactions with the other people or with the environment. As one example, the sensory eyewear system can recognize and interpret a sign language, and present the translated information to a user of the mixed reality device. The wearable system can also recognize text in the user's environment, modify the text (e.g., by changing the content or display characteristics of the text), and render the modified text to occlude the original text.

Abstract: An optical device includes a waveguide including an in-coupling optical element configured to in-couple light into the waveguide, a light distributing element configured to receive light from the in-coupling optical element and distribute light at a selected wavelength, and an out-coupling optical element configured to receive light from the light distributing element and out-couple light out of the waveguide. The out-coupling optical element includes a first region configured to out-couple light at a first depth plane based on a lens function of the first region and a second region configured to out-couple light at a second depth plane based on a different lens function of the second region.