Abstract: An apparatus configured to be head-worn by a user, includes: a screen configured to present graphics for the user; a camera system configured to view an environment in which the user is located; and a processing unit coupled to the camera system, the processing unit configured to: obtain locations of features for an image of the environment, wherein the locations of the features are identified by a neural network; determine a region of interest for one of the features in the image, the region of interest having a size that is less than a size of the image; and perform a corner detection using a corner detection algorithm to identify a corner in the region of interest.

Abstract: An eyepiece waveguide for an augmented reality display system. The eyepiece waveguide can include an input coupling grating (ICG) region. The ICG region can couple an input beam into the substrate of the eyepiece waveguide as a guided beam. A first combined pupil expander-extractor (CPE) grating region can be formed on or in a surface of the substrate. The first CPE grating region can receive the guided beam, create a first plurality of diffracted beams at a plurality of distributed locations, and out-couple a first plurality of output beams. The eyepiece waveguide can also include a second CPE grating region formed on or in the opposite surface of the substrate. The second CPE grating region can receive the guided beam, create a second plurality of diffracted beams at a plurality of distributed locations, and out-couple a second plurality of output beams.

Abstract: An apparatus for providing gaze tracking in a near-eye display. Certain examples provide an apparatus including a light modulator configured to receive light of a first range of wavelengths and generate an image beam therefrom. The light modulator is further configured to receive light of a second range of wavelengths and generate a probe beam therefrom. The apparatus also includes one or more light guides including one or more in-coupling element areas, and one or more out-coupling element areas. The one or more in-coupling diffractive element areas are configured to receive and in-couple the image beam and the probe beam into the one or more light guides. The one or more out-coupling element areas are configured to out-couple, from the one or more light guides: the image beam to a user's eye for user viewing, and the probe beam to the user's eye for detection of reflection therefrom.

Abstract: Cross reality (XR) display devices, such as augmented reality devices, with enhanced state control for anchor-based augmented reality applications are disclosed. In some embodiments, the devices are configured to obtain information identifying applications with respective anchor locations located within a first threshold distance metric of an XR device, with an anchor location corresponding to a real-world location at which virtual content is to be presented; determining respective states, selected from a multitude of states, to be assigned to the applications, the states being determined based on a proximity of the anchor locations to the XR device; implementing the states, with a first application being assigned a state to render virtual content, and with the first application presenting virtual content via the XR device; and in response to movement of the XR device, determining updated states of the one or more applications.

Abstract: In some embodiments, a system comprises a head-mounted frame removably coupleable to the user's head; one or more light sources coupled to the head-mounted frame and configured to emit light with at least two different wavelengths toward a target object in an irradiation field of view of the light sources; one or more electromagnetic radiation detectors coupled to the head-mounted member and configured to receive light reflected after encountering the target object; and a controller operatively coupled to the one or more light sources and detectors and configured to determine and display an output indicating the identity or property of the target object as determined by the light properties measured by the detectors in relation to the light properties emitted by the light sources.

Abstract: Examples of eye-imaging apparatus using diffractive optical elements are provided. For example, an optical device comprises a substrate having a proximal surface and a distal surface, a first coupling optical element disposed on one of the proximal and distal surfaces of the substrate, and a second coupling optical element disposed on one of the proximal and distal surfaces of the substrate and offset from the first coupling optical element. The first coupling optical element can be configured to deflect light at an angle to totally internally reflect (TIR) the light between the proximal and distal surfaces and toward the second coupling optical element, and the second coupling optical element can be configured to deflect at an angle out of the substrate. The eye-imaging apparatus can be used in a head-mounted display such as an augmented or virtual reality display.

Abstract: A method and system for increasing dynamic digitized wavefront resolution, i.e., the density of output beamlets, can include receiving a single collimated source light beam and producing multiple output beamlets spatially offset when out-coupled from a waveguide. The multiple output beamlets can be obtained by offsetting and replicating a collimated source light beam. Alternatively, the multiple output beamlets can be obtained by using a collimated incoming source light beam having multiple input beams with different wavelengths in the vicinity of the nominal wavelength of a particular color. The collimated incoming source light beam can be in-coupled into the eyepiece designed for the nominal wavelength. The input beams with multiple wavelengths take different paths when they undergo total internal reflection in the waveguide, which produces multiple output beamlets.

Abstract: Examples of an imaging system for use with a head mounted display (HMD) are disclosed. The imaging system can include a forward-facing imaging camera and a surface of a display of the HMD can include an off-axis diffractive optical element (DOE) or hot mirror configured to reflect light to the imaging camera. The DOE or hot mirror can be segmented. The imaging system can be used for eye tracking, biometric identification, multiscopic reconstruction of the three-dimensional shape of the eye, etc.

Abstract: A sensory eyewear system for a mixed reality device can facilitate user's interactions with the other people or with the environment. As one example, the sensory eyewear system can recognize and interpret a sign language, and present the translated information to a user of the mixed reality device. The wearable system can also recognize text in the user's environment, modify the text (e.g., by changing the content or display characteristics of the text), and render the modified text to occlude the original text.

Abstract: An electronic device is disclosed. The electronic device comprises a first clock configured to operate at a frequency. First circuitry of the electronic device is configured to synchronize with the first clock. Second circuitry is configured to determine a second clock based on the first clock. The second clock is configured to operate at the frequency of the first clock, and is further configured to operate with a phase shift with respect to the first clock. Third circuitry is configured to synchronize with the second clock.

Abstract: Configurations are disclosed for a health system to be used in various healthcare applications, e.g., for patient diagnostics, monitoring, and/or therapy. The health system may comprise a light generation module to transmit light or an image to a user, one or more sensors to detect a physiological parameter of the user's body, including their eyes, and processing circuitry to analyze an input received in response to the presented images to determine one or more health conditions or defects.

Abstract: Systems and methods for eye pose identification using features of an eye are described. Embodiments of the systems and methods can include segmenting an iris of an eye in the eye image to obtain pupillary and limbic boundaries of the eye, determining two angular coordinates (e.g., pitch and yaw) of an eye pose using the pupillary and limbic boundaries of the eye, identifying an eye feature of the eye (e.g., an iris feature or a scleral feature), determining a third angular coordinate (e.g., roll) of the eye pose using the identified eye feature, and utilizing the eye pose measurement for display of an image or a biometric application. In some implementations, iris segmentation may not be performed, and the two angular coordinates are determined from eye features.

Abstract: A cross reality system that renders virtual content generated by executing native mode applications may be configured to render web-based content using components that render content from native applications. The system may include a Prism manager that provides Prisms in which content from executing native applications is rendered. For rendering web based content, a browser, accessing the web based content, may be associated with a Prism and may render content into its associated Prism, creating the same immersive experience for the user as when content is generated by a native application. The user may access the web application from the same program launcher menu as native applications. The system may have tools that enable a user to access these capabilities, including by creating for a web location an installable entity that, when processed by the system, results in an icon for the web content in a program launcher menu.

Abstract: A method of efficiently and accurately computing a pose of an image with respect to other image information. The image may be acquired with a camera on a portable device and the other information may be a map, such that the computation of pose localizes the device relative to the map. Such a technique may be applied in a cross reality system to enable devices to efficiently and accurately access previously persisted maps. Localizing with respect to a map may enable multiple cross reality devices to render virtual content at locations specified in relation to those maps, providing an enhanced experience for uses of the system. The method may be used in other devices and for other purposes, such as for navigation of autonomous vehicles.

Abstract: An augmented reality display system comprises a passable world model data comprises a set of map points corresponding to one or more objects of the real world. The augmented reality system also comprises a processor to communicate with one or more individual augmented reality display systems to pass a portion of the passable world model data to the one or more individual augmented reality display systems, wherein the piece of the passable world model data is passed based at least in part on respective locations corresponding to the one or more individual augmented reality display systems.

Abstract: An apparatus configured to be worn on a head of a user, includes: a screen configured to present graphics to the user; a camera system configured to view an environment in which the user is located; and a processing unit configured to determine a map based at least in part on output(s) from the camera system, wherein the map is configured for use by the processing unit to localize the user with respect to the environment; wherein the processing unit of the apparatus is also configured to obtain a metric indicating a likelihood of success to localize the user using the map, and wherein the processing unit is configured to obtain the metric by computing the metric or by receiving the metric.