Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for scalable three-dimensional (3-D) object recognition in a cross reality system. One of the methods includes maintaining object data specifying objects that have been recognized in a scene. A stream of input images of the scene is received, including a stream of color images and a stream of depth images. A color image is provided as input to an object recognition system. A recognition output that identifies a respective object mask for each object in the color image is received. A synchronization system determines a corresponding depth image for the color image. A 3-D bounding box generation system determines a respective 3-D bounding box for each object that has been recognized in the color image. Data specifying one or more 3-D bounding boxes is received as output from the 3-D bounding box generation system.

Abstract: Disclosed herein is a mixed reality application to use a multi-channel audio input to identify a character and origin of a given sound, then present a visual representation of the given sound on a near eye display. The visual representation including a vector to the source of the sound. The visual representation further including graphical elements that describe various attributes of the given sound including the magnitude, directionality, source, and threat level. Where the source of the given sound is moving, the visual representation shifts to illustrate the movement.

Abstract: An eyepiece for use in front of an eye of a viewer includes a waveguide having a surface and a diffractive optical element (DOE) optically coupled to the waveguide. The DOE includes a plurality of first ridges protruding from the surface of the waveguide and arranged as a periodic array having a period, each respective first ridge has a first height and a respective first width. The DOE also includes a plurality of second ridges, each respective second ridge protruding from a respective first ridge and having a second height greater than the first height and a respective second width less than the respective first width. At least one of the respective first width, the respective second width, or a respective ratio between the respective first width and the respective second width varies as a function of a distance from a first edge of the DOE.

Abstract: A distributed, cross reality system efficiently and accurately compares location information that includes image frames. Each of the frames may be represented as a numeric descriptor that enables identification of frames with similar content. The resolution of the descriptors may vary for different computing devices in the distributed system based on degree of ambiguity in image comparisons and/or computing resources for the device. A descriptor computed for a cloud-based component operating on maps of large areas that can result in ambiguous identification of multiple image frames may use high resolution descriptors. High resolution descriptors reduce computationally intensive disambiguation processing. A portable device, which is more likely to operate on smaller maps and less likely to have the computational resources to compute a high resolution descriptor, may use a lower resolution descriptor.

Abstract: Disclosed are dimming assemblies and display systems for reducing artifacts produced by optically-transmissive displays. A system may include a substrate upon which a plurality of electronic components are disposed. The electronic components may include a plurality of pixels, a plurality of conductors, and a plurality of circuit modules. The plurality of pixels may be arranged in a two-dimensional array, with each pixel having a two-dimensional geometry corresponding to a shape with at least one curved side. The plurality of conductors may be arranged adjacent to the plurality of pixels. The system may also include control circuitry electrically coupled to the plurality of conductors. The control circuitry may be configured to apply electrical signals to the plurality of circuit modules by way of the plurality of conductors.

Abstract: A method is disclosed, the method comprising the steps of receiving, from a first client application, first graphical data comprising a first node; receiving, from a second client application independent of the first client application, second graphical data comprising a second node; and generating a scenegraph, wherein the scenegraph describes a hierarchical relationship between the first node and the second node according to visual occlusion relative to a perspective from a display.

Abstract: Examples of systems and methods for rendering an avatar in a mixed reality environment are disclosed. The systems and methods may be configured to automatically scale an avatar or to render an avatar based on a determined intention of a user, an interesting impulse, environmental stimuli, or user saccade points. The disclosed systems and methods may apply discomfort curves when rendering an avatar. The disclosed systems and methods may provide a more realistic interaction between a human user and an avatar.

Abstract: A time of flight based depth detection system is disclosed that includes a projector configured to sequentially emit multiple complementary illumination patterns. A sensor of the depth detection system is configured to capture the light from the illumination patterns reflecting off objects within the sensor's field of view. The data captured by the sensor can be used to filter out erroneous readings caused by light reflecting off multiple surfaces prior to returning to the sensor.

Abstract: In some embodiments, optical systems with a reflector and a lens proximate a light output opening of the reflector provide light output with high spatial uniformity and high efficiency. The reflectors are shaped to provide substantially angularly uniform light output and the lens is configured to transform this angularly uniform light output into spatially uniform light output. The light output may be directed into a spatial light modulator, which modulates the light to project an image.

Abstract: Display devices include waveguides with in-coupling optical elements that mitigate re-bounce of in-coupled light to improve overall in-coupling efficiency and/or uniformity. A waveguide receives light from a light source and/or projection optics and includes an in-coupling optical element that in-couples the received light to propagate by total internal reflection in a propagation direction within the waveguide. Once in-coupled into the waveguide the light may undergo re-bounce, in which the light reflects off a waveguide surface and, after the reflection, strikes the in-coupling optical element. Upon striking the in-coupling optical element, the light may be partially absorbed and/or out-coupled by the optical element, thereby effectively reducing the amount of in-coupled light propagating through the waveguide.

Abstract: A wearable display system includes an eyepiece stack having a world side and a user side opposite the world side, wherein during use a user positioned on the user side views displayed images delivered by the system via the eyepiece stack which augment the user's view of the user's environment. The wearable display system also includes an angularly selective film arranged on the world side of the of the eyepiece stack. The angularly selective film includes a polarization adjusting film arranged between pair of linear polarizers. The linear polarizers and polarization adjusting film significantly reduces transmission of visible light incident on the angularly selective film at large angles of incidence without significantly reducing transmission of light incident on the angularly selective film at small angles of incidence.

Abstract: In some embodiments, an augmented reality system includes at least one waveguide that is configured to receive and redirect light toward a user, and is further configured to allow ambient light from an environment of the user to pass therethrough toward the user. The augmented reality system also includes a first adaptive lens assembly positioned between the at least one waveguide and the environment, a second adaptive lens assembly positioned between the at least one waveguide and the user, and at least one processor operatively coupled to the first and second adaptive lens assemblies. Each lens assembly of the augmented reality system is selectively switchable between at least two different states in which the respective lens assembly is configured to impart at least two different optical powers to light passing therethrough, respectively.

Abstract: Embodiments shifts the color fields of a rendered image frame based on the eye tracking data (e.g. position of the user's pupils). An MR device obtains a first image frame having a set of color fields. The first image frame corresponds to a first position of the pupils of the viewer. The MR device then determines a second position of the pupils of the viewer based on, for example, data receive from an eye tracking device coupled to the MR device. The MR device generates, based on the first image frame, a second image frame corresponding to the second position of the pupils. The second image frame is generated by shifting color fields by a shift value based on the second position of the pupils of the viewer. The MR device transmits the second image frame to a display device of the MR device to be displayed thereon.

Abstract: Examples of the disclosure describe systems and methods for presenting an audio signal to a user of a wearable head device. According to an example method, a first input audio signal is received. The first input audio signal is processed to generate a first output audio signal. The first output audio signal is presented via one or more speakers associated with the wearable head device. Processing the first input audio signal comprises applying a pre-emphasis filter to the first input audio signal; adjusting a gain of the first input audio signal; and applying a de-emphasis filter to the first audio signal. Applying the pre-emphasis filter to the first input audio signal comprises attenuating a low frequency component of the first input audio signal. Applying the de-emphasis filter to the first input audio signal comprises attenuating a high frequency component of the first input audio signal.

Abstract: Systems and methods of generating a three-dimensional (3D) reconstruction of a scene or environment surrounding a user of a spatial computing system, such as a virtual reality, augmented reality or mixed reality system, using only multiview images comprising RGB images, and without the need for depth sensors or depth data from sensors. Features are extracted from a sequence of frames of RGB images and back-projected using known camera intrinsics and extrinsic s into a 3D voxel volume wherein each pixel of the voxel volume is mapped to a ray in the voxel volume. The back-projected features are fused into the 3D voxel volume. The 3D voxel volume is passed through a 3D convolutional neural network to refine the and regress truncated signed distance function values at each voxel of the 3D voxel volume.

Abstract: Systems and methods of disabling user control interfaces during attachment of a wearable electronic device to a portion of a user's clothing or accessory are disclosed. The wearable electronic device can include inertial measurement units (IMUs), optical sources, optical sensors or electromagnetic sensors. Based on the information provided by the IMUs, optical sources, optical sensors or electromagnetic sensors, an electrical processing and control system can make a determination that the electronic device is being grasped and picked up for attaching to a portion of a user's clothing or accessory or that the electronic device is in the process of being attached to a portion of a user's clothing or accessory and temporarily disable one or more user control interfaces disposed on the outside of the wearable electronic device.

Abstract: A fixture used in the manufacture of an eyepiece, to cut the eyepiece to a particular shape, and a method of using the fixture to cut the eyepiece to have a desired shape. Embodiments are directed to a configurable fixture to align, hold, and protect a plastic sheet (e.g., a wafer) while a laser cutting apparatus is cutting one or more eyepieces out of the wafer. During the cutting, the fixture protects the eyepieces from reflected laser light by providing voids around the laser cutting lines, and by supporting each eyepiece near its perimeter. The fixture can be quickly rearranged for different eyepieces, different eyepiece shapes, and/or different plastic sheet sizes.

Abstract: Methods for singulating an optical waveguide material at a contour include directing a first laser beam onto a first side of the optical waveguide material to generate a first group of perforations in the optical waveguide material. A second laser beam is directed onto a second side of the optical waveguide material to generate a second group of perforations in the optical waveguide material. The second side is opposite the first side. The first group of perforations and the second group of perforations define a perforation zone at the contour. A third laser beam is directed at the perforation zone to singulate the optical waveguide material at the perforation zone.

Abstract: An optical device includes a liquid crystal layer having a first plurality of liquid crystal molecules arranged in a first pattern and a second plurality of liquid crystal molecules arranged in a second pattern. The first and the second pattern are separated from each other by a distance of about 20 nm and about 100 nm along a longitudinal or a transverse axis of the liquid crystal layer. The first and the second plurality of liquid crystal molecules are configured as first and second grating structures that can redirect light of visible or infrared wavelengths.

Abstract: An optical combiner includes a curved reflective element and a rotating mirror configured to rotate through a range of angular displacement. During a first time period, the curved reflective element is configured to reflect a first light beam emitted from a first light source to the rotating mirror when the rotating mirror is disposed at a first angular displacement, and the rotating mirror is configured to receive the first reflected light beam and provide a first output light beam along an output optical axis. During a second time period, the curved reflective element is configured to reflect a second light beam emitted from a second light source to the rotating mirror when the rotating mirror is disposed at a second angular displacement, and the rotating mirror is configured to receive the second reflected light beam and provide a second output light beam along the output optical axis.