Abstract: A method of switching between augmented-reality (AR) user interface states at an AR headset. The method includes, causing the AR headset to present a glance user interface element at a first portion of a display of the AR headset. The method further includes, in response to a first request from a user of the AR headset to display a follow user interface element, causing the AR headset to present the follow user interface element at a second portion of the display, wherein the second portion is larger than the third portion. The method further includes, in response to a second request from the user of the AR headset to display a home user interface element, causing the AR headset to present the home user interface element at a third portion of the display, wherein the third portion is larger than the second portion.

Abstract: Aspects of the present disclosure are directed to providing an augmented world environment model for an artificial reality (XR) environment. Some implementations of the present technology define how experiences will run on an XR system as a hierarchy of entities, including experiences, augments those experiences create, and an exclusive mode in which a particular experience takes control of the entire view. Some implementations can establish a framework for multiple experiences to interface with each other and to run in an XR ecosystem. This can improve upon traditional mobile and desktop operating systems by providing different states optimized for XR, which can include improved utilization of resources on an XR device, such as an XR head-mounted display. Experiences and augments can enter a particular state based on criteria, and can be, based on the state, processed differently by an operating system or provided with particular capabilities.

Abstract: A method for automatic switching between gaze tracking and hand tracking is described. The method occurs while an extended-reality (XR) headset is worn by a user. The method includes obtaining gaze data captured at the XR headset. The method further includes determining, based on the gaze data, a first point of focus within an XR interface presented at a display of the XR headset. The method further includes, in accordance with a determination that the gaze data does not satisfy a gaze-quality threshold: (i) obtaining image data captured at the XR headset indicating a projected-point position of a hand of the user within the XR interface, and (ii) determining, based on the image data and the projected-point position, a second point of focus within the XR interface presented by the display of the XR headset.

Abstract: Aspects of the disclosure are directed to an interface for receiving input using multiple modalities in an artificial reality environment. The interface can be a virtual keyboard displayed in an artificial reality environment that includes characters arranged as elements. Implementations include an artificial reality device/system for displaying the artificial reality environment and receiving user input in a first modality, and a controller device for receiving user input in an additional input modality. For example, the artificial reality system can be configured to receive user gaze input as a first input modality and the controller device can be configured to receive input in a second modality, such as touch input received at a trackpad. An interface manager can process input in the multiple modalities to control an indicator on the virtual interface. The interface manager can also resolve character selections from the virtual interface according to the input.

Abstract: Aspects of the present disclosure are directed to transitioning an artificial reality (XR) experience, of multiple XR experiences, from operating in a shared experience environment to operating in an exclusive mode, where other non-selected XR experiences are hidden or otherwise placed into an inactive state during the transitioning process. When an XR experience operates in the shared experience environment, the XR experience can be limited by or otherwise constrained by one or more rules of the shared experience environment—such as a specified area into which the XR experience can write content. When an XR experience transitions to exclusive mode, the selected XR experience can access resources that are otherwise unavailable to the selected XR experience in the shared experience environment. For example, transitioned XR experience can access a greater amount of processing capacity, display real estate, and/or memory of the XR device.

Abstract: Aspects of the present disclosure are directed to providing an augmented world environment model for an artificial reality (XR) environment. Some implementations of the present technology define how experiences will run on an XR system as a hierarchy of entities, including experiences, augments those experiences create, and an exclusive mode in which a particular experience takes control of the entire view. Some implementations can establish a framework for multiple experiences to interface with each other and to run in an XR ecosystem. This can improve upon traditional mobile and desktop operating systems by providing different states optimized for XR, which can include improved utilization of resources on an XR device, such as an XR head-mounted display. Experiences and augments can enter a particular state based on criteria, and can be, based on the state, processed differently by an operating system or provided with particular capabilities.

Abstract: Aspects of the disclosure are directed to an interface for receiving input using multiple modalities in an artificial reality environment. The interface can be a virtual keyboard displayed in an artificial reality environment that includes characters arranged as elements. Implementations include an artificial reality device/system for displaying the artificial reality environment and receiving user input in a first modality, and a controller device for receiving user input in an additional input modality. For example, the artificial reality system can be configured to receive user gaze input as a first input modality and the controller device can be configured to receive input in a second modality, such as touch input received at a trackpad. An interface manager can process input in the multiple modalities to control an indicator on the virtual interface. The interface manager can also resolve character selections from the virtual interface according to the input.

Abstract: Aspects of the disclosure are directed to an interface for receiving input using multiple modalities in an artificial reality environment. The interface can be a virtual keyboard displayed in an artificial reality environment that includes characters arranged as elements. Implementations include an artificial reality device/system for displaying the artificial reality environment and receiving user input in a first modality, and a controller device for receiving user input in an additional input modality. For example, the artificial reality system can be configured to receive user gaze input as a first input modality and the controller device can be configured to receive input in a second modality, such as touch input received at a trackpad. An interface manager can process input in one or more of the modalities to control an indicator on the virtual interface. The interface manager can also resolve character selections from the virtual interface according to the input.