Abstract: Aspects of the present disclosure are directed to providing an artificial reality environment with augments and surfaces. An “augment” is a virtual container in 3D space that can include presentation data, context, and logic. An artificial reality system can use augments as the fundamental building block for displaying 2D and 3D models in the artificial reality environment. For example, augments can represent people, places, and things in an artificial reality environment and can respond to a context such as a current display mode, time of day, a type of surface the augment is on, a relationship to other augments, etc. Augments can be on a “surface” that has a layout and properties that cause augments on that surface to display in different ways. Augments and other objects (real or virtual) can also interact, where these interactions can be controlled by rules for the objects evaluated based on information from the shell.

Abstract: In one embodiment, a method includes instructing sensors on an AR headset to capture data of a real-world environment, discovering a 3D location where a native application configured to run natively on an operating system is downloadable, generating and displaying a virtual object representing the native application at the 3D location, detecting a user of the AR system interacting with the virtual object, downloading a first executable portion of the native application, executing the first executable portion within a sandbox process having a predetermined set of access permissions, progressively downloading a remaining executable portion of the native application while the user is interacting with the executed first executable portion, and in response to a determination that the native application is completely downloaded, installing the native application on the operating system.

Abstract: A virtual object system can orchestrate virtual objects defined as a collection of components and with inheritance in an object hierarchy. Virtual object components can include a container, data, a template, and a controller. A container can define the volume the virtual object is authorized to write into. A virtual object's data can specify features such as visual elements, parameters, links to external data, meta-data, etc. The template can define view states of the virtual object and contextual breakpoints for transitioning between them. Each view state can control when and how the virtual object presents data elements. The controller can define logic for the virtual object to respond to input, context, etc. The definition of each object can specify which other object in an object hierarchy that object extends, where extending an object includes inheriting that object's components, which can be modified or overwritten as part of the extension.

Abstract: In one embodiment, a method includes instructing sensors on an AR headset to capture data of a real-world environment, discovering a 3D location where a native application configured to run natively on an operating system is downloadable, generating and displaying a virtual object representing the native application at the 3D location, detecting a user of the AR system interacting with the virtual object, downloading a first executable portion of the native application, executing the first executable portion within a sandbox process having a predetermined set of access permissions, progressively downloading a remaining executable portion of the native application while the user is interacting with the executed first executable portion, and in response to a determination that the native application is completely downloaded, installing the native application on the operating system.

Abstract: An artificial reality environment (XRE) schema is defined that supports controlling interactions between various artificial reality actors. The XRE schema includes a set of definitions for an XRE, independent of type of artificial reality device. The definitions in the XRE schema can include standards for both interfaces and data objects. The XRE schema can define XR elements in terms of entities and components of a space, organized according to a hierarchy. Each entity can represent a real or virtual object or space, within the XRE, defined by a name and a collection of one or more components. Each component (as part of an entity) can define aspects and expose information about the entity. The XRE schema can specify structures that that allow actors (e.g., producers, instantiators, and consumers) to define and perform actions in relation to XRE elements.

Abstract: A virtual object system can orchestrate virtual objects defined as a collection of components and with inheritance in an object hierarchy. Virtual object components can include a container, data, a template, and a controller. A container can define the volume the virtual object is authorized to write into. A virtual object's data can specify features such as visual elements, parameters, links to external data, meta-data, etc. The template can define view states of the virtual object and contextual breakpoints for transitioning between them. Each view state can control when and how the virtual object presents data elements. The controller can define logic for the virtual object to respond to input, context, etc. The definition of each object can specify which other object in an object hierarchy that object extends, where extending an object includes inheriting that object's components, which can be modified or overwritten as part of the extension.

Abstract: The present disclosure provides world-controlled augments and application-controlled augments. World-controlled augments can be controlled directly by a shell in the artificial reality environment. To allow even inexperienced users to develop world-controlled augments, a world-controlled builder system is provided. Application-controlled augments may be resource intensive (e.g., using eye-tracking, social-media tie-ins, etc.), may support complicated interactions among themselves, may require or have extensive use of inputs and permissioned resources, and are controlled by their hosting application. When a running application is halted, the application closes its application-controlled augments but can choose to have the XR system run a “place-holder” world-controlled augment for as long as the application is not running. The place-holder world-controlled augment preserves the appearance of the application in the artificial reality environment but uses few system resources.

Abstract: Aspects of the present disclosure are directed to providing an artificial reality environment with augments and surfaces. An “augment” is a virtual container in 3D space that can include presentation data, context, and logic. An artificial reality system can use augments as the fundamental building block for displaying 2D and 3D models in the artificial reality environment. For example, augments can represent people, places, and things in an artificial reality environment and can respond to a context such as a current display mode, time of day, a type of surface the augment is on, a relationship to other augments, etc. Augments can be on a “surface” that has a layout and properties that cause augments on that surface to display in different ways. Augments and other objects (real or virtual) can also interact, where these interactions can be controlled by rules for the objects evaluated based on information from the shell.

Abstract: The present disclosure provides world-controlled augments and application-controlled augments. World-controlled augments can be controlled directly by a shell in the artificial reality environment. To allow even inexperienced users to develop world-controlled augments, a world-controlled builder system is provided. Application-controlled augments may be resource intensive (e.g., using eye-tracking, social-media tie-ins, etc.), may support complicated interactions among themselves, may require or have extensive use of inputs and permissioned resources, and are controlled by their hosting application. When a running application is halted, the application closes its application-controlled augments but can choose to have the XR system run a “place-holder” world-controlled augment for as long as the application is not running. The place-holder world-controlled augment preserves the appearance of the application in the artificial reality environment but uses few system resources.

Abstract: In one embodiment, a method includes, by an operating system of a first artificial-reality device, receiving a notification that virtual objects are shared with the first artificial-reality device by a second artificial-reality device, where the virtual objects are shared by being placed inside a sender-side shared space anchored to a physical object. The method further includes the first artificial-reality device accessing descriptors of a physical object and a spatial-relationship definition between the physical object and a receiver-side shared space, detecting physical objects based on the descriptors, determining pose of the receiver-side shared space, detecting physical constraints within the receiver-side shares space, receiving display instructions for the virtual objects, and rendering the virtual objects on the first artificial-reality device in the receiver-side shared space.

Abstract: An artificial reality environment (XRE) schema is defined that supports anel-controlling interactions between various artificial reality actors. The XRE schema includes a set of definitions for an XRE, independent of type of artificial reality device. The definitions in the XRE schema can include standards for both interfaces and data objects. The XRE schema can define XR elements in terms of entities and components of a space, organized according to a hierarchy. Each entity can represent a real or virtual object or space, within the XRE, defined by a name and a collection of one or more components. Each component (as part of an entity) can define aspects and expose information about the entity. The XRE schema can specify structures that that allow actors (e.g., producers, instantiators, and consumers) to define and perform actions in relation to XRE elements.

Abstract: Aspects of the present disclosure are directed to an artificial reality system orchestrating interactions between virtual object “augments.” The orchestration can include linking, which can be forming two or more augments into a combination, embedding an augment within an existing combination, or triggering an action mapped to the linking of those augments. Another type of orchestration can include extracting, which can refer to taking an augment out of an existing combination, either by removing it from the combination or copying the augment to leave a version in the combination and having another version outside the combination.

Abstract: In one embodiment, a method includes, by an operating system of a first artificial-reality device, receiving a notification that virtual objects are shared with the first artificial-reality device by a second artificial-reality device, where the virtual objects are shared by being placed inside a sender-side shared space anchored to a physical object. The method further includes the first artificial-reality device accessing descriptors of a physical object and a spatial-relationship definition between the physical object and a receiver-side shared space, detecting physical objects based on the descriptors, determining pose of the receiver-side shared space, detecting physical constraints within the receiver-side shares space, receiving display instructions for the virtual objects, and rendering the virtual objects on the first artificial-reality device in the receiver-side shared space.

Abstract: A virtual object system can orchestrate virtual objects defined as a collection of components and with inheritance in an object hierarchy. Virtual object components can include a container, data, a template, and a controller. A container can define the volume the virtual object is authorized to write into. A virtual object's data can specify features such as visual elements, parameters, links to external data, meta-data, etc. The template can define view states of the virtual object and contextual breakpoints for transitioning between them. Each view state can control when and how the virtual object presents data elements. The controller can define logic for the virtual object to respond to input, context, etc. The definition of each object can specify which other object in an object hierarchy that object extends, where extending an object includes inheriting that object's components, which can be modified or overwritten as part of the extension.

Abstract: Aspects of the present disclosure are directed to an artificial reality system orchestrating interactions between virtual object “augments.” The orchestration can include linking, which can be forming two or more augments into a combination, embedding an augment within an existing combination, or triggering an action mapped to the linking of those augments. Another type of orchestration can include extracting, which can refer to taking an augment out of an existing combination, either by removing it from the combination or copying the augment to leave a version in the combination and having another version outside the combination.

Abstract: Aspects of the present disclosure are directed to providing an artificial reality environment with augments and surfaces. An “augment” is a virtual container in 3D space that can include presentation data, context, and logic. An artificial reality system can use augments as the fundamental building block for displaying 2D and 3D models in the artificial reality environment. For example, augments can represent people, places, and things in an artificial reality environment and can respond to a context such as a current display mode, time of day, a type of surface the augment is on, a relationship to other augments, etc. Augments can be on a “surface” that has a layout and properties that cause augments on that surface to display in different ways. Augments and other objects (real or virtual) can also interact, where these interactions can be controlled by rules for the objects evaluated based on information from the shell.

Abstract: The present disclosure provides world-controlled augments and application-controlled augments. World-controlled augments can be controlled directly by a shell in the artificial reality environment. To allow even inexperienced users to develop world-controlled augments, a world-controlled builder system is provided. Application-controlled augments may be resource intensive (e.g., using eye-tracking, social-media tie-ins, etc.), may support complicated interactions among themselves, may require or have extensive use of inputs and permissioned resources, and are controlled by their hosting application. When a running application is halted, the application closes its application-controlled augments but can choose to have the XR system run a “place-holder” world-controlled augment for as long as the application is not running. The place-holder world-controlled augment preserves the appearance of the application in the artificial reality environment but uses few system resources.

Abstract: A virtual object system can orchestrate virtual objects defined as a collection of components and with inheritance in an object hierarchy. Virtual object components can include a container, data, a template, and a controller. A container can define the volume the virtual object is authorized to write into. A virtual object's data can specify features such as visual elements, parameters, links to external data, meta-data, etc. The template can define view states of the virtual object and contextual breakpoints for transitioning between them. Each view state can control when and how the virtual object presents data elements. The controller can define logic for the virtual object to respond to input, context, etc. The definition of each object can specify which other object in an object hierarchy that object extends, where extending an object includes inheriting that object's components, which can be modified or overwritten as part of the extension.

Abstract: In some implementations, the disclosed systems and methods can automatically generate seller listing titles and descriptions for products; set a follow-me mode for various virtual objects. In some implementations, an artificial reality device can generate a virtual reality environment where a user authorizes a potential spectator as a registered spectator by registering an account of the potential spectator with an account of the user. In some implementations, the disclosed systems and methods can capture video from within a virtual reality (VR) environment using one or more virtual cameras.

Abstract: Aspects of the present disclosure are directed to providing an artificial reality environment with augments and surfaces. An “augment” is a virtual container in 3D space that can include presentation data, context, and logic. An artificial reality system can use augments as the fundamental building block for displaying 2D and 3D models in the artificial reality environment. For example, augments can represent people, places, and things in an artificial reality environment and can respond to a context such as a current display mode, time of day, a type of surface the augment is on, a relationship to other augments, etc. Augments can be on a “surface” that has a layout and properties that cause augments on that surface to display in different ways. Augments and other objects (real or virtual) can also interact, where these interactions can be controlled by rules for the objects evaluated based on information from the shell.