Abstract: Aspects of the present disclosure are directed to artificial reality (XR) controllers and hand gestures and corresponding representation of the hand gestures in XR space. Users can benefit from being able to use touch interactions while also holding the XR controller. Example implementations allow a user to use controllers for distance interaction and interactions that require controller buttons, but also recognize when the user extends her finger and wants to manually touch things—using touch in conjunction with controllers. Thus, if the user has a controller in her hand and want to touch a UI element, she can also use her finger, where the system identifies the finger not touching the controller (e.g., via capacitance sensors) and that a hand pose is present (e.g., finger extended—from camera) to allow that finger to provide hand input, without the user having to put down the controller.

Abstract: Aspects of the present disclosure are directed to artificial reality (XR) controllers and hand gestures and corresponding representation of the hand gestures in XR space. Users can benefit from being able to use touch interactions while also holding the XR controller. Example implementations allow a user to use controllers for distance interaction and interactions that require controller buttons, but also recognize when the user extends her finger and wants to manually touch things—using touch in conjunction with controllers. Thus, if the user has a controller in her hand and want to touch a UI element, she can also use her finger, where the system identifies the finger not touching the controller (e.g., via capacitance sensors) and that a hand pose is present (e.g., finger extended—from camera) to allow that finger to provide hand input, without the user having to put down the controller.

Abstract: Aspects of the present disclosure provide a quick actions menu in an artificial reality (XR) environment that can be accessed by performing a hand gesture detected by an XR device which, in some implementations, can be detected without the use of controllers. Once the menu is open, the user can move her hand while performing the gesture to highlight a particular quick action and can release the gesture on a highlighted action to select the action. In some implementations, the user can drill down into an action on the menu by highlighting the action, then dragging the gesture off of the action away from the menu. To close the quick actions menu, the user can either A) move the gesture off the menu and release the gesture, B) rotate the wrist, or C) explicitly dismiss the menu, such as by using a voice command detected by the XR device.

Abstract: Aspects of the present disclosure are directed to a two-dimensional (2D) user interface (UI) content overlay for an artificial reality (XR) environment. In some implementations, the content overlay can be a default home screen when a user launches into an immersive XR application that helps the user navigate and control immersive destinations, and complete relevant and frequent actions related to the XR application. In some implementations, the content overlay can serve dynamic content based on the user's context to help them, e.g., A) find and play with other users, B) learn relevant actions that can be taken, C) quickly travel to a destination, D) see relevant content, etc. The content overlay can provide these functions 2D cards in the XR environment, which can be specific to the executing XR application, or can be system-level for multiple providing content XR applications.

Abstract: An artificial reality (XR) system includes XR controllers and hand gestures and corresponding representation of the hand gestures in XR space. Example implementations allow a user to use controllers for distance interaction and interactions that require controller buttons, but also recognize when the user extends her finger and wants to manually touch things-using touch in conjunction with controllers. Thus, if the user has a controller in her hand and wants to touch a UI element, she can also use her finger. In particular, the system identifies the finger is not touching the controller (e.g., via capacitance sensors) and that a hand pose is present (e.g., finger extended—from camera), thereby processing hand input from the finger without the user having to put down the controller.

Abstract: Aspects of the present disclosure are directed to artificial reality (XR) controllers and hand gestures and corresponding representation of the hand gestures in XR space. Users can benefit from being able to use touch interactions while also holding the XR controller. Example implementations allow a user to use controllers for distance interaction and interactions that require controller buttons, but also recognize when the user extends her finger and wants to manually touch things-using touch in conjunction with controllers. Thus, if the user has a controller in her hand and want to touch a UI element, she can also use her finger, where the system identifies the finger not touching the controller (e.g., via capacitance sensors) and that a hand pose is present (e.g., finger extended—from camera) to allow that finger to provide hand input, without the user having to put down the controller.