Abstract: Configurations are disclosed for presenting virtual reality and augmented reality experiences to users. An augmented reality display system comprises a handheld component housing an electromagnetic field emitter, the electromagnetic field emitter emitting a known magnetic field, the head mounted component coupled to one or more electromagnetic sensors that detect the magnetic field emitted by the electromagnetic field emitter housed in the handheld component, wherein a head pose is known, and a controller communicatively coupled to the handheld component and the head mounted component, the controller receiving magnetic field data from the handheld component, and receiving sensor data from the head mounted component, wherein the controller determining a hand pose based at least in part on the received magnetic field data and the received sensor data.

Abstract: Methods and systems for manufacturing an optical waveguide include depositing an adhesion promoting layer on a substrate. Multiple curable resist droplets are dispensed on the adhesion promoting layer. The adhesion promoting layer is disposed between and contacts the substrate and the curable resist droplets. The curable resist droplets define an optical eyepiece layer such that a zero residual layer thickness (RLT) region of the optical eyepiece layer is free of the curable resist droplets. The optical eyepiece layer is incised from the substrate to form the optical waveguide.

Abstract: A cross reality system enables any of multiple devices to efficiently and accurately access previously persisted maps, even maps of very large environments, and render virtual content specified in relation to those maps. The cross reality system may quickly process a batch of images acquired with a portable device to determine whether there is sufficient consistency across the batch in the computed localization. Processing on at least one image from the batch may determine a rough localization of the device to the map. This rough localization result may be used in a refined localization process for the image for which it was generated. The rough localization result may also be selectively propagated to a refined localization process for other images in the batch, enabling rough localization processing to be skipped for the other images.

Abstract: Augmented reality systems and methods for creating, saving and rendering designs comprising multiple items of virtual contenting a three-dimensional (3D) environment of a user. The designs may be saved as a scene, which is built by a user from pre-built sub-components, built components, and/or previously saved scenes. Location information, expressed as a saved scene anchor and position relative to the saved scene anchor for each item of virtual content, may also be saved. Upon opening the scene, the saved scene anchor node may be correlated to a location within the mixed reality environment of the user for whom the scene is opened. The virtual items of the scene may be positioned with the same relationship to that location as they have to the saved scene anchor node. That location may be selected automatically and/or by user input.

Abstract: An image sensor suitable for use in an augmented reality system to provide low latency image analysis with low power consumption. The augmented reality system can be compact, and may be small enough to be packaged within a wearable device such as a set of goggles or mounted on a frame resembling ordinary eyeglasses. The image sensor may receive information about a region of an imaging array associated with a movable object and selectively output imaging information for that region. The region may be updated dynamically as the image sensor and/or the object moves. Such an image sensor provides a small amount of data from which object information used in rendering an augmented reality scene can be developed. The amount of data may be further reduced by configuring the image sensor to output indications of pixels for which the measured intensity of incident light changes.

Abstract: A computer implemented method of facilitating communication between first and second users includes displaying, by a first head-worn device, a first virtual object to the first user first user wearing the first head-worn device. The method also includes displaying, by a second head-worn device, a second virtual object to the second user wearing the second head-worn device. The method further includes facilitating, by the first and second head-worn devices, communications between the first and second users using the first and second virtual objects to simulate the first and second users being present in a common environment.

Abstract: One embodiment is directed to a user display device comprising a housing frame mountable on the head of the user, a lens mountable on the housing frame and a projection sub system coupled to the housing frame to determine a location of appearance of a display object in a field of view of the user based at least in part on at least one of a detection of a head movement of the user and a prediction of a head movement of the user, and to project the display object to the user based on the determined location of appearance of the display object.

Abstract: An augmented reality headset includes a projector and an eyepiece waveguide stack optically coupled to the projector. The eyepiece waveguide stack includes: a first eyepiece waveguide including a first incoupling diffractive optical element and a first combined pupil expander; and a second eyepiece waveguide including a second incoupling diffractive optical element and a second combined pupil expander. The second incoupling diffractive optical element is laterally offset from the first incoupling diffractive optical element in a lateral direction.

Abstract: A cross reality system enables any of multiple devices to efficiently and accurately access previously persisted maps of very large scale environments and render virtual content specified in relation to those maps. The cross reality system may quickly determine whether a 2D set of features derived from images acquired with a portable device match a set of 3D features of an environment map and, if so, determine the relative pose of the feature sets. The pose may be used in quickly and accurately localizing the portable device to the environment map. Pairs of features in the 2D and 3D features sets may be identified based on matching feature descriptors and may be scored in a neural network trained to assess the quality of the match. Poses may be identified based on subsets of the matching features weighted towards pairs of features with high quality.

Abstract: An augmented reality system includes a projector and projection optics coupled to the projector. The augmented reality system also includes an eyepiece waveguide including hybrid diffractive structure including: one or more first nanofeatures having first material with a first index of refraction; and one or more second nanofeatures having a second material with a second index of refraction.

Abstract: A wearable display system for a cross reality (XR) system may have a dynamic vision sensor (DVS) camera and a color camera. At least one of the cameras may be a plenoptic camera. The wearable display system may dynamically restrict processing of image data from either or both cameras based on detected conditions and XR function being performed. For tracking an object, image information may be processed for patches of a field of view of either or both cameras corresponding to the object. The object may be tracked based on asynchronously acquired events indicating changes within the patches. Stereoscopic or other types of image information may be used when event-based object tacking yields an inadequate quality metric. The tracked object may be a user's hand or a stationary object in the physical world, enabling calculation of the pose of the wearable display system and of the wearer's head.

Abstract: A method of fabricating a blazed diffraction grating comprises providing a master template substrate and imprinting periodically repeating lines on the master template substrate in a plurality of master template regions. In some embodiments, the periodically repeating lines in different ones of the master template regions extend in different directions. The method additionally comprises using at least one of the master template regions as a master template to imprint at least one blazed diffraction grating pattern on a grating substrate. In some embodiments, the method further comprises coating the periodically repeating lines with a material having a greater hardness than the material that forms the lines.

Abstract: Display devices include waveguides with metasurfaces as in-coupling and/or out-coupling optical elements. The metasurfaces may be formed on a surface of the waveguide and may include a plurality or an array of sub-wavelength-scale (e.g., nanometer-scale) protrusions. Individual protrusions may include horizontal and/or vertical layers of different materials which may have different refractive indices, allowing for enhanced manipulation of light redirecting properties of the metasurface. Some configurations and combinations of materials may advantageously allow for broadband metasurfaces. Manufacturing methods described herein provide for vertical and/or horizontal layers of different materials in a desired configuration or profile.

Abstract: An apparatus including a set of three illumination sources disposed in a first plane. Each of the set of three illumination sources is disposed at a position in the first plane offset from others of the set of three illumination sources by 120 degrees measured in polar coordinates. The apparatus also includes a set of three waveguide layers disposed adjacent the set of three illumination sources. Each of the set of three waveguide layers includes an incoupling diffractive element disposed at a lateral position offset by 180 degrees from a corresponding illumination source of the set of three illumination sources.

Abstract: A wearable device includes a left optical stack having a left eyepiece configured to receive left virtual image light, a left accommodating lens, and a left compensating lens. The wearable device also includes a right optical stack having a right eyepiece configured to receive right virtual image light, a right accommodating lens, and a right compensating lens. An optical power of the left accommodating lens is equal in magnitude to an optical power of the left compensating lens, an optical power of the right accommodating lens is equal in magnitude to an optical power of the right compensating lens, and the optical power of the left accommodating lens and the optical power of the right accommodating lens differ by an offset amount.

Abstract: Several unique hardware configurations and methods for freeform optical display systems are disclosed. A freeform display system includes primary freeform optical element(s) and secondary freeform optical element(s) in tiled arrangements to expand the horizontal field of view (FOV) or the vertical field of view. The system may include a variable focusing system that produces intermediate pupil and changes the focal distance of a single focal plane or switches among multiple focal planes for rendering objects in focus while resolving accommodation-convergence conflict. The system may map light samples to appropriate light rays in physical space and use a cluster of projectors to project the mapped light rays to produce the light field of the virtual display content. Methods for making tiled freeform optical display systems and methods for producing virtual content with variable focus freeform optics and rendering light fields are also disclosed.

Abstract: Disclosed are methods and systems for a scripting framework and implementations therefor for mixed reality software applications of heterogeneous systems. These methods or systems create a mixed-reality software application that executes across heterogeneous platforms on a server-side instance of a scripting framework and manage a change in the mixed-reality software application using the server-side instance of the scripting framework. Moreover, the change in the mixed-reality software application using a client-side instance of the scripting framework; and the mixed-reality software application may be interactively executed on a mixed-reality device.

Abstract: Disclosed is an improved diffraction structure for 3D display systems. The improved diffraction structure includes an intermediate layer that resides between a waveguide substrate and a top grating surface. The top grating surface comprises a first material that corresponds to a first refractive index value, the underlayer comprises a second material that corresponds to a second refractive index value, and the substrate comprises a third material that corresponds to a third refractive index value. According to additional embodiments, improved approaches are provided to implement deposition of imprint materials onto a substrate, which allow for very precise distribution and deposition of different imprint patterns onto any number of substrate surfaces.

Abstract: Plasma etching processes for forming patterns in high refractive index glass substrates, such as for use as waveguides, are provided herein. The substrates may be formed of glass having a refractive index of greater than or equal to about 1.65 and having less than about 50 wt % SiO2. The plasma etching processes may include both chemical and physical etching components. In some embodiments, the plasma etching processes can include forming a patterned mask layer on at least a portion of the high refractive index glass substrate and exposing the mask layer and high refractive index glass substrate to a plasma to remove high refractive index glass from the exposed portions of the substrate. Any remaining mask layer is subsequently removed from the high refractive index glass substrate. The removal of the glass forms a desired patterned structure, such as a diffractive grating, in the high refractive index glass substrate.