Abstract: Techniques are disclosed for performing localization of a handheld device with respect to a wearable device. At least one sensor mounted to the handheld device, such as an inertial measurement unit (IMU), may obtain handheld data indicative of movement of the handheld device with respect to the world. An imaging device mounted to either the handheld device or the wearable device may capture a fiducial image containing a number of fiducials affixed to the other device. The number of fiducials contained in the image are determined. Based on the number of fiducials, at least one of a position and an orientation of the handheld device with respect to the wearable device are updated based on the image and the handheld data in accordance with a first operating state, a second operating state, or a third operating state.

Abstract: Disclosed herein are systems and methods for distributed computing and/or networking for mixed reality systems. A method may include capturing an image via a camera of a head-wearable device. Inertial data may be captured via an inertial measurement unit of the head-wearable device. A position of the head-wearable device can be estimated based on the image and the inertial data via one or more processors of the head-wearable device. The image can be transmitted to a remote server. A neural network can be trained based on the image via the remote server. A trained neural network can be transmitted to the head-wearable device.

Abstract: An artifact mitigation system includes a projector assembly and a set of imaging optics optically coupled to the projector assembly. The artifact mitigation system also includes an eyepiece optically coupled to the set of imaging optics. The eyepiece includes a diffractive incoupling interface. The artifact mitigation system further includes an artifact prevention element disposed between the set of imaging optics and the eyepiece. The artifact prevention element includes a linear polarizer, a first quarter waveplate disposed adjacent the linear polarizer, and a color select component disposed adjacent the first quarter waveplate.

Abstract: The invention provides a content provisioning system. A mobile device has a mobile device processor. The mobile device mobile device has communication interface connected to the mobile device processor and a first resource device communication interface and under the control of the mobile device processor to receive first content transmitted by the first resource device transmitter The mobile device mobile device has a mobile device output device connected to the mobile device processor and under control of the mobile device processor capable of providing an output that can be sensed by a user.

Abstract: Disclosed herein are systems and methods for distributed computing and/or networking for mixed reality systems. A method may include capturing an image via a camera of a head-wearable device. Inertial data may be captured via an inertial measurement unit of the head-wearable device. A position of the head-wearable device can be estimated based on the image and the inertial data via one or more processors of the head-wearable device. The image can be transmitted to a remote server. A neural network can be trained based on the image via the remote server. A trained neural network can be transmitted to the head-wearable device.

Abstract: The invention provides a content provisioning system. A mobile device has a mobile device processor. The mobile device mobile device has communication interface connected to the mobile device processor and a first resource device communication interface and under the control of the mobile device processor to receive first content transmitted by the first resource device transmitter The mobile device mobile device has a mobile device output device connected to the mobile device processor and under control of the mobile device processor capable of providing an output that can be sensed by a user.

Abstract: An artifact mitigation system includes a projector assembly and a set of imaging optics optically coupled to the projector assembly. The artifact mitigation system also includes an eyepiece optically coupled to the set of imaging optics. The eyepiece includes a diffractive incoupling interface. The artifact mitigation system further includes an artifact prevention element disposed between the set of imaging optics and the eyepiece. The artifact prevention element includes a linear polarizer, a first quarter waveplate disposed adjacent the linear polarizer, and a color select component disposed adjacent the first quarter waveplate.

Abstract: Disclosed herein are systems and methods for distributed computing and/or networking for mixed reality systems. A method may include capturing an image via a camera of a head-wearable device. Inertial data may be captured via an inertial measurement unit of the head-wearable device. A position of the head-wearable device can be estimated based on the image and the inertial data via one or more processors of the head-wearable device. The image can be transmitted to a remote server. A neural network can be trained based on the image via the remote server. A trained neural network can be transmitted to the head-wearable device.

Abstract: A sensory eyewear system for a mixed reality device can facilitate user's interactions with the other people or with the environment. As one example, the sensory eyewear system can recognize and interpret a sign language, and present the translated information to a user of the mixed reality device. The wearable system can also recognize text in the user's environment, modify the text (e.g., by changing the content or display characteristics of the text), and render the modified text to occlude the original text.

Abstract: An optical device includes a waveguide including an in-coupling optical element configured to in-couple light into the waveguide, a light distributing element configured to receive light from the in-coupling optical element and distribute light at a selected wavelength, and an out-coupling optical element configured to receive light from the light distributing element and out-couple light out of the waveguide. The out-coupling optical element includes a first region configured to out-couple light at a first depth plane based on a lens function of the first region and a second region configured to out-couple light at a second depth plane based on a different lens function of the second region.

Abstract: An example head-mounted display device includes a light projector and an eyepiece. The eyepiece includes a light guiding layer and a first focusing optical element. The first focusing optical element includes a first region having a first optical power, and a second region having a second optical power different from the first optical power. The light guiding layer is configured to: i) receive light from the light projector, ii) direct at least a first portion of the light to a user's eye through the first region to present a first virtual image to the user at a first focal distance, and iii) direct at least a second portion of the light to the user's eye through the second region to present a second virtual image to the user at a second focal distance.

Abstract: Techniques are disclosed for performing localization of a handheld device with respect to a wearable device. At least one sensor mounted to the handheld device, such as an inertial measurement unit (IMU), may obtain handheld data indicative of movement of the handheld device with respect to the world. An imaging device mounted to either the handheld device or the wearable device may capture a fiducial image containing a number of fiducials affixed to the other device. The number of fiducials contained in the image are determined. Based on the number of fiducials, at least one of a position and an orientation of the handheld device with respect to the wearable device are updated based on the image and the handheld data in accordance with a first operating state, a second operating state, or a third operating state.

Abstract: An eyepiece for an augmented reality display system. The eyepiece can include a waveguide substrate. The waveguide substrate can include an input coupler grating (ICG), an orthogonal pupil expander (OPE) grating, a spreader grating, and an exit pupil expander (EPE) grating. The ICG can couple at least one input light beam into at least a first guided light beam that propagates inside the waveguide substrate. The OPE grating can divide the first guided light beam into a plurality of parallel, spaced-apart light beams. The spreader grating can receive the light beams from the OPE grating and spread their distribution. The spreader grating can include diffractive features oriented at approximately 90° to diffractive features of the OPE grating. The EPE grating can re-direct the light beams from the first OPE grating and the first spreader grating such that they exit the waveguide substrate.

Abstract: An example head-mounted display device includes a light projector and an eyepiece. The eyepiece includes a light guiding layer and a first focusing optical element. The first focusing optical element includes a first region having a first optical power, and a second region having a second optical power different from the first optical power. The light guiding layer is configured to: i) receive light from the light projector, ii) direct at least a first portion of the light to a user's eye through the first region to present a first virtual image to the user at a first focal distance, and iii) direct at least a second portion of the light to the user's eye through the second region to present a second virtual image to the user at a second focal distance.

Abstract: Disclosed herein are systems and methods for distributed computing and/or networking for mixed reality systems. A method may include capturing an image via a camera of a head-wearable device. Inertial data may be captured via an inertial measurement unit of the head-wearable device. A position of the head-wearable device can be estimated based on the image and the inertial data via one or more processors of the head-wearable device. The image can be transmitted to a remote server. A neural network can be trained based on the image via the remote server. A trained neural network can be transmitted to the head-wearable device.

Abstract: An eyepiece waveguide for an augmented reality. The eyepiece waveguide can include a transparent substrate with an input coupler region, a first orthogonal pupil expander (OPE) region, and an exit pupil expander (EPE) region. The input coupler region can couple an input light beam that is externally incident on the input coupler region into at least a first guided light beam that propagates inside the substrate. The first OPE region can divide the first guided beam into a plurality of replicated, spaced-apart beams. The EPE region can re-direct the replicated beams from the first OPE region such that they exit the substrate. The EPE region can have an amount of optical power.

Abstract: An eyepiece for a head-mounted display includes one or more first waveguides arranged to receive light from a spatial light modulator at a first edge, guide at least some of the received light to a second edge opposite the first edge, and extract at least some of the light through a face of the one or more first waveguides between the first and second edges. The eyepiece also includes a second waveguide positioned to receive light exiting the one or more first waveguides at the second edge and guide the received light to one or more light absorbers.

Abstract: Techniques are disclosed for performing localization of a handheld device with respect to a wearable device. At least one sensor mounted to the handheld device, such as an inertial measurement unit (IMU), may obtain handheld data indicative of movement of the handheld device with respect to the world. An imaging device mounted to either the handheld device or the wearable device may capture a fiducial image containing a number of fiducials affixed to the other device. The number of fiducials contained in the image are determined. Based on the number of fiducials, at least one of a position and an orientation of the handheld device with respect to the wearable device are updated based on the image and the handheld data in accordance with a first operating state, a second operating state, or a third operating state.

Abstract: An eyepiece for a head-mounted display includes one or more first waveguides arranged to receive light from a spatial light modulator at a first edge, guide at least some of the received light to a second edge opposite the first edge, and extract at least some of the light through a face of the one or more first waveguides between the first and second edges. The eyepiece also includes a second waveguide positioned to receive light exiting the one or more first waveguides at the second edge and guide the received light to one or more light absorbers.

Abstract: An eyepiece for projecting an image includes a first planar waveguide positioned in a first lateral plane. The first planar waveguide comprises a first diffractive optical element (DOE) disposed at a first lateral position. The eyepiece also includes a second planar waveguide positioned in a second lateral plane adjacent to the first lateral plane. The second planar waveguide comprises a second DOE disposed at a second lateral position different from the first lateral position. The eyepiece further includes a third planar waveguide positioned in a third lateral plane adjacent to the second lateral plane. The third planar waveguide comprises a third DOE disposed at a third lateral position different from the first lateral position and the second lateral position. The eyepiece also includes an optical filter positioned between the second planar waveguide and the third planar waveguide. The optical filter is disposed at the third lateral position.