Abstract: In accordance with some embodiments, an exemplary process for controlling representations of virtual objects based on respective user contexts that each correspond to different respective locations in a computer-generated reality (CGR) environment is described.

Abstract: The present disclosure relates to techniques for displaying representations of physical input devices and overlaying visual features on the representations of physical input devices in a computer-generated reality (CGR) environment. The techniques include displaying a virtual application in a CGR environment and, in response to detecting an input field in the displayed virtual application, displaying at least a portion of the displayed application on a representation of a physical input device. The at least a portion of the displayed application includes the detected input field. In response to detecting an input received at the physical input device, the input field is updated with the input, and the updated input field is displayed.

Abstract: Exemplary processes are described, including processes to move and/or resize user interface elements in a computer-generated reality environment.

Abstract: In some implementations, the disclosed systems and methods can detect an interaction with respect to a set of virtual objects, which can start with a particular gesture, and take an action with respect to one or more virtual objects based on a further interaction (e.g., holding the gesture for a particular amount of time, moving the gesture in a particular direction, releasing the gesture, etc.). In some implementations, the disclosed systems and methods can automatically review a 3D video to determine a depicted user or avatar movement pattern (e.g., dance moves, repair procedure, playing an instrument, etc.). In some implementations, the disclosed systems and methods can allow the gesture to included a flat hand with the user's thumb next to the palm, with the gesture toward the user's face.

Abstract: A virtual object system can orchestrate virtual objects defined as a collection of components and with inheritance in an object hierarchy. Virtual object components can include a container, data, a template, and a controller. A container can define the volume the virtual object is authorized to write into. A virtual object's data can specify features such as visual elements, parameters, links to external data, meta-data, etc. The template can define view states of the virtual object and contextual breakpoints for transitioning between them. Each view state can control when and how the virtual object presents data elements. The controller can define logic for the virtual object to respond to input, context, etc. The definition of each object can specify which other object in an object hierarchy that object extends, where extending an object includes inheriting that object's components, which can be modified or overwritten as part of the extension.

Abstract: In accordance with some embodiments, an exemplary process for dynamically controlling the size of a display based on a moving of a visual object meeting a criterion in a computer-generated reality (CGR) environment is described.

Abstract: Various implementations disclosed herein include devices, systems, and methods that enable presenting environments including visual representations of multiple applications. In one implementation, a method includes presenting a view of an environment at an electronic device on a display. The view includes visual representations of a plurality of applications. The method further includes determining to provide a first application with access to a control parameter. The control parameter is configured to modify at least a portion of the view of the environment with virtual content, and the portion of the view includes at least a portion of content outside of a view of a visual representation associated with the first application. The method further includes restricting access to the control parameter by other applications which prevents the other applications from modifying the at least the portion of the view of the environment via the control parameter.

Abstract: In accordance with some embodiments, an exemplary process for controlling representations of virtual objects based on respective user contexts that each correspond to different respective locations in a computer-generated reality (CGR) environment is described.

Abstract: The present disclosure provides world-controlled augments and application-controlled augments. World-controlled augments can be controlled directly by a shell in the artificial reality environment. To allow even inexperienced users to develop world-controlled augments, a world-controlled builder system is provided. Application-controlled augments may be resource intensive (e.g., using eye-tracking, social-media tie-ins, etc.), may support complicated interactions among themselves, may require or have extensive use of inputs and permissioned resources, and are controlled by their hosting application. When a running application is halted, the application closes its application-controlled augments but can choose to have the XR system run a “place-holder” world-controlled augment for as long as the application is not running. The place-holder world-controlled augment preserves the appearance of the application in the artificial reality environment but uses few system resources.

Abstract: A virtual object system can orchestrate virtual objects defined as a collection of components and with inheritance in an object hierarchy. Virtual object components can include a container, data, a template, and a controller. A container can define the volume the virtual object is authorized to write into. A virtual object's data can specify features such as visual elements, parameters, links to external data, meta-data, etc. The template can define view states of the virtual object and contextual breakpoints for transitioning between them. Each view state can control when and how the virtual object presents data elements. The controller can define logic for the virtual object to respond to input, context, etc. The definition of each object can specify which other object in an object hierarchy that object extends, where extending an object includes inheriting that object's components, which can be modified or overwritten as part of the extension.

Abstract: In accordance with some embodiments, an exemplary process for dynamically controlling the size of a display based on a moving of a visual object meeting a criterion in a computer-generated reality (CGR) environment is described.

Abstract: Aspects of the disclosure are directed to an interface for receiving input using multiple modalities in an artificial reality environment. The interface can be a virtual keyboard displayed in an artificial reality environment that includes characters arranged as elements. Implementations include an artificial reality device/system for displaying the artificial reality environment and receiving user input in a first modality, and a controller device for receiving user input in an additional input modality. For example, the artificial reality system can be configured to receive user gaze input as a first input modality and the controller device can be configured to receive input in a second modality, such as touch input received at a trackpad. An interface manager can process input in the multiple modalities to control an indicator on the virtual interface. The interface manager can also resolve character selections from the virtual interface according to the input.

Abstract: Aspects of the present disclosure are directed to triggering virtual keyboard selections using multiple input modalities. An interface manager can display an interface, such as a virtual keyboard, to a user in an artificial reality environment. Implementations of the interface manager can track user eye gaze input and user hand input (e.g., hand or finger motion). The interface manager can resolve a character selection on the virtual keyboard according to the tracked user gaze input based on detection that the user's hand motion meets a trigger criteria. For example, the interface manager can: detect that the tracked user hand motion meets the trigger criteria at a given point in time; and resolve a selection from the virtual keyboard (e.g., selection of a displayed character) according to the tracked user gaze on the virtual keyboard at the given point in time.

Abstract: Aspects of the disclosure are directed to an interface for receiving input using multiple modalities in an artificial reality environment. The interface can be a virtual keyboard displayed in an artificial reality environment that includes characters arranged as elements. Implementations include an artificial reality device/system for displaying the artificial reality environment and receiving user input in a first modality, and a controller device for receiving user input in an additional input modality. For example, the artificial reality system can be configured to receive user gaze input as a first input modality and the controller device can be configured to receive input in a second modality, such as touch input received at a trackpad. An interface manager can process input in one or more of the modalities to control an indicator on the virtual interface. The interface manager can also resolve character selections from the virtual interface according to the input.

Abstract: Various implementations disclosed herein include devices, systems, and methods that enable presenting environments including visual representations of multiple applications. In one implementation, a method includes presenting a view of an environment at an electronic device on a display. The view includes visual representations of a plurality of applications. The method further includes determining to provide a first application with access to a control parameter. The control parameter is configured to modify at least a portion of the view of the environment with virtual content, and the portion of the view includes at least a portion of content outside of a view of a visual representation associated with the first application. The method further includes restricting access to the control parameter by other applications which prevents the other applications from modifying the at least the portion of the view of the environment via the control parameter.

Abstract: A virtual object system can orchestrate virtual objects defined as a collection of components and with inheritance in an object hierarchy. Virtual object components can include a container, data, a template, and a controller. A container can define the volume the virtual object is authorized to write into. A virtual object's data can specify features such as visual elements, parameters, links to external data, meta-data, etc. The template can define view states of the virtual object and contextual breakpoints for transitioning between them. Each view state can control when and how the virtual object presents data elements. The controller can define logic for the virtual object to respond to input, context, etc. The definition of each object can specify which other object in an object hierarchy that object extends, where extending an object includes inheriting that object's components, which can be modified or overwritten as part of the extension.

Abstract: The present disclosure relates to techniques for displaying representations of physical input devices and overlaying visual features on the representations of physical input devices in a computer-generated reality (CGR) environment. The techniques include displaying a virtual application in a CGR environment and, in response to detecting an input field in the displayed virtual application, displaying at least a portion of the displayed application on a representation of a physical input device. The at least a portion of the displayed application includes the detected input field. In response to detecting an input received at the physical input device, the input field is updated with the input, and the updated input field is displayed.

Abstract: In some embodiments, an electronic device navigates between user interfaces based at least on detecting a gaze of the user. In some embodiments, an electronic device enhances interactions with control elements of user interfaces. In some embodiments, an electronic device scrolls representations of categories and subcategories in a coordinated manner. In some embodiments, an electronic device navigates back from user interfaces having different levels of immersion in different ways.

Abstract: A virtual object system can orchestrate virtual objects defined as a collection of components and with inheritance in an object hierarchy. Virtual object components can include a container, data, a template, and a controller. A container can define the volume the virtual object is authorized to write into. A virtual object's data can specify features such as visual elements, parameters, links to external data, meta-data, etc. The template can define view states of the virtual object and contextual breakpoints for transitioning between them. Each view state can control when and how the virtual object presents data elements. The controller can define logic for the virtual object to respond to input, context, etc. The definition of each object can specify which other object in an object hierarchy that object extends, where extending an object includes inheriting that object's components, which can be modified or overwritten as part of the extension.

Abstract: Controller devices can provide a user with multiple modes of operation. For example, the user can hold and operate the controller device in a first orientation for control of a first device and/or first mode of a device (e.g., media player), and the user can hold and operate the controller device in a second orientation for control of a second device and/or second mode of a device (e.g., gaming device). Features can be selectively stowed and deployed to facilitate user operation in the different modes. By further example, a controller device can be separated into segments to allow multiple users to hold and provide inputs with corresponding segments.