Abstract: Various implementations disclosed herein include devices, systems, and methods that enable multi-mode interactions with elements in a three-dimensional (3D) environment based on cursor movement associated with tracking user hand motion. For example, a process may include presenting an extended reality (XR) environment comprising a virtual element and a cursor. The process may further include obtaining hand data corresponding to 3D movement of a hand in a 3D environment. The process may further include operating in first mode where the 3D motion of the hand is converted to two-dimensional (2D) motion and detecting a 3D user input criteria. In response to the 3D user input criteria a mode of operation is modified to a second mode where the 3D motion of the hand is maintained without conversion to the 2D motion.

Abstract: Some techniques are described herein for integrating a 2D framework with a 3D framework. Such techniques use a concept referred to as a hidden entity to link the two frameworks together. Other techniques are described herein for translating gestures from a first type to a second type in certain situations.

Abstract: Out-of-process hit-testing can provide increased privacy and efficiency in computer user input systems. In an aspect, an application specifies a control style for a UI window to be managed separately from the application, such as by a system process running outside of the application process. When a user input is received at a location corresponding to a part of the application UI that is separate from the UI window for which the control style has been specified, the user input may be redirected to the UI window for which the control style has been specified.

Abstract: Implementations of the subject technology provide virtual anchoring for extended reality (XR) display devices. A device may generate an XR environment that includes computer-generated (CG) content for display relative to various physical objects in a physical environment. In order to position the CG content, an XR application may request a physical anchor object to which the CG content can be anchored. In circumstances in which the physical anchor object is not available in the physical environment, a virtual anchor and/or a virtual anchor object corresponding to the physical anchor object can be provided to which the CG content can be anchored.

Abstract: Out-of-process hit-testing can provide increased privacy and efficiency in computer user input systems. In an aspect, an application specifies a control style for a UI window to be managed separately from the application, such as by a system process running outside of the application process. When a user input is received at a location corresponding to a part of the application UI that is separate from the UI window for which the control style has been specified, the user input may be redirected to the UI window for which the control style has been specified.

Abstract: Various implementations disclosed herein include devices, systems, and methods that transport a cursor between surfaces of objects within an XR environment. For example, an example process may include displaying a movement of a cursor across a first surface of a first object in a view of a three-dimensional (3D) environment. The process may further include determining that movement of the cursor approaches or intersects a boundary of the first surface at a first position. The process may further include determining a second position on a second surface of a second object in the 3D environment based on a path of the cursor. The process may further include moving the cursor from the first position to the second position.

Abstract: Implementations of the subject technology provide virtual anchoring for extended reality (XR) display devices. A device may generate an XR environment that includes computer-generated (CG) content for display relative to various physical objects in a physical environment. In order to position the CG content, an XR application may request a physical anchor object to which the CG content can be anchored. In circumstances in which the physical anchor object is not available in the physical environment, a virtual anchor and/or a virtual anchor object corresponding to the physical anchor object can be provided to which the CG content can be anchored.

Abstract: A computer system displays a first object that includes at least a first portion of the first object and a second portion of the first object and detects a first gaze input that meets first criteria, wherein the first criteria require that the first gaze input is directed to the first portion of the first object in order for the first criteria to be met. In response, the computer system displays a first control element that corresponds to a first operation associated with the first object, wherein the first control element was not displayed prior to detecting that the first gaze input met the first criteria, and detects a first user input directed to the first control element. In response to detecting the first user input directed to the first control element, the computer system performs the first operation with respect to the first object.

Abstract: A head-mounted device is provided that includes displays configured to display an image and to simultaneously display a magnifying window that presents a magnified portion of the image. The magnifying window lies in a magnification plane that is fixed relative to a user's head. One or more processors in the head-mounted device can be used to perform a first ray cast operation to identify an input point where a detected user input intersects the magnifying window, to obtain a remapped point from the input point, to compute a directional vector based on the remapped point and a reference point associated with the user's head, to obtain a shifted point by shifting the remapped point from the magnification plane to another plane parallel to the magnification plane, and to perform a second ray cast operation using the shifted point and the directional vector.

Abstract: While a view of an environment is visible via a display generation component of a computer system, the computer system detects a gaze input directed to a first location, corresponding to a first user interface element, in the environment. In response to detecting the gaze input: if a user's hand is in a predefined configuration during the gaze input, the computer system: provides, to the first user interface element, information about the gaze input; and then, in response to detecting the gaze input moving to a different, second location in the environment while the user's hand is maintained in the predefined configuration, provides, to a second user interface element that corresponds to the second location, information about the gaze input. If the user's hand is not in the predefined configuration during the gaze input, the computer system forgoes providing, to the first user interface element, information about the gaze input.

Abstract: Out-of-process hit-testing can provide increased privacy and efficiency in computer user input systems. In an aspect, an application specifies a control style for a UI window to be managed separately from the application, such as by a system process running outside of the application process. When a user input is received at a location corresponding to a part of the application UI that is separate from the UI window for which the control style has been specified, the user input may be redirected to the UI window for which the control style has been specified.

Abstract: Various implementations disclosed herein more accurately or efficiently determine to which of multiple potential virtual objects user input should be directed in a 3D graphical environment. In some implementations, this involves using a rule that accounts for the types of the virtual objects to which a particular event may correspond. For example, a direction of intent may be identified and a rule used to determine to which of multiple potential virtual objects to associate an event.

Abstract: Aspects of the subject technology relate to gaze-based control of an electronic device. The gaze-based control can include enabling an option to provide user authorization when it is determined that the user has viewed and/or read text associated with a request for the user authorization. The gaze-based control can also include modifying a user interface or a user interface element based on user views and/or reads. The gaze-based control can be based on determining whether a user has viewed and/or read an electronic document and/or a physical document.

Abstract: Implementations of the subject technology provide virtual anchoring for extended reality (XR) display devices. A device may generate an XR environment that includes computer-generated (CG) content for display relative to various physical objects in a physical environment. In order to position the CG content, an XR application may request a physical anchor object to which the CG content can be anchored. In circumstances in which the physical anchor object is not available in the physical environment, a virtual anchor and/or a virtual anchor object corresponding to the physical anchor object can be provided to which the CG content can be anchored.

Abstract: Implementations of the subject technology provide continuous transfer of content editing and/or control between various devices in an extended reality system. The extended reality system includes at least one device that is capable of determining the locations of other devices in the system. This device can manage continuous transfer of control between other devices in the system responsive to three-dimensional location-based user inputs, and/or can manage continuous transfer of control between one or more of the other devices and the device itself.