Abstract: An augmented reality display system includes a pair of variable focus lens elements that sandwich a waveguide stack. One of the lens elements is positioned between the waveguide stack and a user's eye to correct for refractive errors in the focusing of light projected from the waveguide stack to that eye. The lens elements may also be configured to provide appropriate optical power to place displayed virtual content on a desired depth plane. The other lens element is between the ambient environment and the waveguide stack, and is configured to provide optical power to compensate for aberrations in the transmission of ambient light through the waveguide stack and the lens element closest to the eye. In addition, an eye-tracking system monitors the vergence of the user's eyes and automatically and continuously adjusts the optical powers of the pair of lens elements based on the determined vergence of those eyes.

Abstract: Configurations are disclosed for presenting virtual reality and augmented reality experiences to users. The system may comprise an image capturing device to capture one or more images, the one or more images corresponding to a field of the view of a user of a head-mounted augmented reality device, and a processor communicatively coupled to the image capturing device to extract a set of map points from the set of images, to identify a set of sparse points and a set of dense points from the extracted set of map points, and to perform a normalization on the set of map points.

Abstract: This disclosure describes a head-mounted display with a display assembly configured to display content to most or all of a user's field of view. The display assembly can be configured to display content in far-peripheral regions of the user's field of view differently than content upon which a user can focus. For example, spatial resolution, color resolution, refresh rate and intensity (i.e. brightness) can be adjusted to save resources and/or to bring attention to virtual content positioned within a far-peripheral region. In some embodiments, these changes can save processing resources without detracting from the user's overall experience.

Abstract: One embodiment is directed to a system for enabling two or more users to interact within a virtual world comprising virtual world data, comprising a computer network comprising one or more computing devices, the one or more computing devices comprising memory, processing circuitry, and software stored at least in part in the memory and executable by the processing circuitry to process at least a portion of the virtual world data; wherein at least a first portion of the virtual world data originates from a first user virtual world local to a first user, and wherein the computer network is operable to transmit the first portion to a user device for presentation to a second user, such that the second user may experience the first portion from the location of the second user, such that aspects of the first user virtual world are effectively passed to the second user.

Abstract: A display system includes a head-mounted display configured to project light, having different amounts of wavefront divergence, to an eye of a user to display virtual image content appearing to be disposed at different depth planes. The wavefront divergence may be changed in discrete steps, with the change in steps being triggered based upon whether the user is fixating on a particular depth plane. The display system may be calibrated for switching depth planes for a main user. Upon determining that a guest user is utilizing the system, rather than undergoing a full calibration, the display system may be configured to switch depth planes based on a rough determination of the virtual content that the user is looking at. The virtual content has an associated depth plane and the display system may be configured to switch to the depth plane of that virtual content.

Abstract: Methods and systems are disclosed for presenting virtual objects on a limited number of depth planes using, e.g., an augmented reality display system. A farthest one of the depth planes is within a mismatch tolerance of optical infinity. The display system may switch the depth plane on which content is actively displayed, so that the content is displayed on the depth plane on which a user is fixating. The impact of errors in fixation tracking is addressed using partially overlapping depth planes. A fixation depth at which a user is fixating is determined and the display system determines whether to adjust selection of a selected depth plane at which a virtual object is presented. The determination may be based on whether the fixation depth falls within a depth overlap region of adjacent depth planes. The display system may switch the active depth plane depending upon whether the fixation depth falls outside the overlap region.

Abstract: A fixed-distance display system includes a light source configured to generate a light beam. The system also includes a light guiding optical element configured to propagate at least a portion of the light beam by total internal reflection. The system further includes a first inertial measurement unit configured to measure a first value for calculating a head pose of a user. Moreover, the system includes a camera configured to capture an image for machine vision optical flow analysis. The display system is configured to display virtual images only within a tolerance range of a single predetermined optical plane.

Abstract: Examples of a light field metrology system for use with a display are disclosed. The light field metrology may capture images of a projected light field, and determine focus depths (or lateral focus positions) for various regions of the light field using the captured images. The determined focus depths (or lateral positions) may then be compared with intended focus depths (or lateral positions), to quantify the imperfections of the display. Based on the measured imperfections, an appropriate error correction may be performed on the light field to correct for the measured imperfections. The display can be an optical display element in a head mounted display, for example, an optical display element capable of generating multiple depth planes or a light field display.

Abstract: Configurations are disclosed for presenting virtual reality and augmented reality experiences to users. The system may comprise an image capturing device to capture one or more images, the one or more images corresponding to a field of the view of a user of a head-mounted augmented reality device, and a processor communicatively coupled to the image capturing device to extract a set of map points from the set of images, to identify a set of sparse points and a set of dense points from the extracted set of map points, and to perform a normalization on the set of map points.

Abstract: A hearing aid comprises a sensor configured for detecting a focus of an end user on a real sound source, a microphone assembly configured for converting sounds into electrical signals, a speaker configured for converting the electrical signals into sounds, and a control subsystem configured for modifying the direction and/or distance of a greatest sensitivity of the microphone assembly based on detected focus. A virtual image generation system comprises memory storing a three-dimensional scene, a sensor configured for detecting a focus of the end user on a sound source, a speaker configured for conveying sounds to the end user, and a control subsystem configured for causing the speaker to preferentially convey a sound originating from the sound source in response to detection of the focus, and for rendering image frames of the scene, and a display subsystem configured for sequentially displaying the image frames to the end user.

Abstract: Methods and systems are disclosed for presenting virtual objects on a limited number of depth planes using, e.g., an augmented reality display system. A farthest one of the depth planes is within a mismatch tolerance of optical infinity. The display system may switch the depth plane on which content is actively displayed, so that the content is displayed on the depth plane on which a user is fixating. The impact of errors in fixation tracking is addressed using partially overlapping depth planes. A fixation depth at which a user is fixating is determined and the display system determines whether to adjust selection of a selected depth plane at which a virtual object is presented. The determination may be based on whether the fixation depth falls within a depth overlap region of adjacent depth planes. The display system may switch the active depth plane depending upon whether the fixation depth falls outside the overlap region.

Abstract: A display system includes a head-mounted display configured to project light, having different amounts of wavefront divergence, to an eye of a user to display virtual image content appearing to be disposed at different depth planes. The wavefront divergence may be changed in discrete steps, with the change in steps being triggered based upon whether the user is fixating on a particular depth plane. The display system may be calibrated for switching depth planes for a main user. Upon determining that a guest user is utilizing the system, rather than undergoing a full calibration, the display system may be configured to switch depth planes based on a rough determination of the virtual content that the user is looking at. The virtual content has an associated depth plane and the display system may be configured to switch to the depth plane of that virtual content.

Abstract: A method of explaining the reasons for a prediction made by a machine learning ensemble prediction process as to the probability of an outcome for a target observation following training on a plurality of training observations determines the similarity between the target observation and each training observation of a set of said training observations; selects a fraction of the training observations that are most similar to said target observation; ranks the training observations by similarity of each training observation to the target observation; and determines the significance of the features of the ranked training observations to the prediction based upon the increase in variance in a local prediction model when a feature is removed from the local model.

Abstract: Examples of a light field metrology system for use with a display are disclosed. The light field metrology may capture images of a projected light field, and determine focus depths (or lateral focus positions) for various regions of the light field using the captured images. The determined focus depths (or lateral positions) may then be compared with intended focus depths (or lateral positions), to quantify the imperfections of the display. Based on the measured imperfections, an appropriate error correction may be performed on the light field to correct for the measured imperfections. The display can be an optical display element in a head mounted display, for example, an optical display element capable of generating multiple depth planes or a light field display.

Abstract: An augmented reality display system comprises a passable world model data comprises a set of map points corresponding to one or more objects of the real world. The augmented reality system also comprises a processor to communicate with one or more individual augmented reality display systems to pass a portion of the passable world model data to the one or more individual augmented reality display systems, wherein the piece of the passable world model data is passed based at least in part on respective locations corresponding to the one or more individual augmented reality display systems.

Abstract: Aspects of the subject disclosure may include, for example, identifying an entity of a natural language question, locating a node of a knowledge graph corresponding to the entity, and generating a candidate answer set including a group of other entities located a predetermined proximity to the node. Contextual information for the group of other entities is determined from the knowledge graph, and the natural language question and contextual information are separately encoded to obtain separate encoded vectorial representations of the natural language question and members of the candidate answer set. The encoding uses pre-trained language model embeddings obtained via a bidirectional encoder representations from transformer encoding process. The encoded vectorial representations of the question under an influence of aspects of the contextual information are scored and a member of the candidate answer set selected according to the score to obtain an answer to the original question.

Abstract: A hearing aid comprises a sensor configured for detecting a focus of an end user on a real sound source, a microphone assembly configured for converting sounds into electrical signals, a speaker configured for converting the electrical signals into sounds, and a control subsystem configured for modifying the direction and/or distance of a greatest sensitivity of the microphone assembly based on detected focus. A virtual image generation system comprises memory storing a three-dimensional scene, a sensor configured for detecting a focus of the end user on a sound source, a speaker configured for conveying sounds to the end user, and a control subsystem configured for causing the speaker to preferentially convey a sound originating from the sound source in response to detection of the focus, and for rendering image frames of the scene, and a display subsystem configured for sequentially displaying the image frames to the end user.

Abstract: A wearable device may include a head-mounted display (HMD) for rendering a three-dimensional (3D) virtual object which appears to be located in an ambient environment of a user of the display. The relative positions of the HMD and one or more eyes of the user may not be in desired positions to receive, or register, image information outputted by the HMD. For example, the HMD-to-eye alignment vary for different users and may change over time (e.g., as a given user moves around or as the HMD slips or otherwise becomes displaced). The wearable device may determine a relative position or alignment between the HMD and the user's eyes. Based on the relative positions, the wearable device may determine if it is properly fitted to the user, may provide feedback on the quality of the fit to the user, and may take actions to reduce or minimize effects of any misalignment.

Abstract: An augmented reality viewing system is described. A local coordinate frame of local content is transformed to a world coordinate frame. A further transformation is made to a head coordinate frame and a further transformation is made to a camera coordinate frame that includes all pupil positions of an eye. One or more users may interact in separate sessions with a viewing system. If a canonical map is available, the earlier map is downloaded onto a viewing device of a user. The viewing device then generates another map and localizes the subsequent map to the canonical map.

Abstract: Techniques are disclosed for performing localization of a handheld device with respect to a wearable device. At least one sensor mounted to the handheld device, such as an inertial measurement unit (IMU), may obtain handheld data indicative of movement of the handheld device with respect to the world. An imaging device mounted to either the handheld device or the wearable device may capture a fiducial image containing a number of fiducials affixed to the other device. The number of fiducials contained in the image are determined. Based on the number of fiducials, at least one of a position and an orientation of the handheld device with respect to the wearable device are updated based on the image and the handheld data in accordance with a first operating state, a second operating state, or a third operating state.