Abstract: To register an object for subsequent identification by an artificial reality headset, the headset receives an input from a user to enter a registration mode. The headset includes a plurality of imaging devices capturing images of a local area surrounding the headset. An eye tracking unit of the headset determines a gaze direction of the user, and the headset identifies an object in the local area where the user's gaze is directed and visually distinguishes the identified object to the user. In response to receiving a confirmation from the user to register the identified object, one or more images of the identified object are captured by the imaging devices and used to train an instance classifier to identify the identified object. The headset notifies the user when the identified object has been registered.

Abstract: Examples of systems and methods to facilitate audiovisual presence transitions of virtual objects such as virtual avatars in a mixed reality collaborative environment are disclosed. The systems and methods may be configured to produce different audiovisual presence transitions such as appearance, disappearance and reappearance of the virtual avatars. The virtual avatar audiovisual transitions may be further indicated by various visual and sound effects of the virtual avatars. The transitions may occur based on various colocation or decolocation scenarios.

Abstract: Examples of systems and methods to facilitate audiovisual presence transitions of virtual objects such as virtual avatars in a mixed reality collaborative environment are disclosed. The systems and methods may be configured to produce different audiovisual presence transitions such as appearance, disappearance and reappearance of the virtual avatars. The virtual avatar audiovisual transitions may be further indicated by various visual and sound effects of the virtual avatars. The transitions may occur based on various colocation or decolocation scenarios.

Abstract: To register an object for subsequent identification by an artificial reality headset, the headset receives an input from a user to enter a registration mode. The headset includes a plurality of imaging devices capturing images of a local area surrounding the headset. An eye tracking unit of the headset determines a gaze direction of the user, and the headset identifies an object in the local area where the user's gaze is directed and visually distinguishes the identified object to the user. In response to receiving a confirmation from the user to register the identified object, one or more images of the identified object are captured by the imaging devices and used to train an instance classifier to identify the identified object. The headset notifies the user when the identified object has been registered.

Abstract: A headset, such as an artificial reality headset, includes a depth camera assembly that generates a three-dimensional model of a local area surrounding the headset. Additionally, the headset identifies objects in the local area through application of one or more trained models to images of the local area captured by imaging devices. The headset uses a bounding box determined for an identified object to map the identified object to the three-dimensional model of the local area. Based on the mapping, the headset may guide the user to the identified object or display content proximate to the identified object through a display element.

Abstract: Examples of systems and methods to facilitate audiovisual presence transitions of virtual objects such as virtual avatars in a mixed reality collaborative environment are disclosed. The systems and methods may be configured to produce different audiovisual presence transitions such as appearance, disappearance and reappearance of the virtual avatars. The virtual avatar audiovisual transitions may be further indicated by various visual and sound effects of the virtual avatars. The transitions may occur based on various colocation or decolocation scenarios.

Abstract: Examples of systems and methods to facilitate audiovisual presence transitions of virtual objects such as virtual avatars in a mixed reality collaborative environment are disclosed. The systems and methods may be configured to produce different audiovisual presence transitions such as appearance, disappearance and reappearance of the virtual avatars. The virtual avatar audiovisual transitions may be further indicated by various visual and sound effects of the virtual avatars. The transitions may occur based on various colocation or decolocation scenarios.

Abstract: Wearable systems for privacy preserving expression generation for augmented or virtual reality client applications. An example method includes receiving, by an expression manager configured to communicate expression information to client applications, a request from a client application for access to the expression information. The expression information reflects information derived from one or more sensors of the wearable system, with the client application being configured to present virtual content including an avatar rendered based on the expression information. A user interface is output for presentation which requests user authorization for the client application to access the expression information. In response to receiving user input indicating user authorization, enabling access to the expression information is enabled. The client application obtains periodic updates to the expression information, and the avatar is rendered based on the periodic updates.

Abstract: Examples of systems and methods for improved hand tracking of a user in a mixed reality environment are disclosed. The systems and methods may be configured to estimate the hand pose and shape of a user's hands for applications such as animating a hand on a user's avatar. Data from multiple sources, such as a totem internal measurement unit (“IMU”), external totem location tracking, vision cameras, and depth sensors, may be manipulated using a set of rules that are based on historical data, ergonomics data, and motion data.

Abstract: Wearable systems for privacy preserving expression generation for augmented or virtual reality client applications. An example method includes receiving, by an expression manager configured to communicate expression information to client applications, a request from a client application for access to the expression information. The expression information reflects information derived from one or more sensors of the wearable system, with the client application being configured to present virtual content including an avatar rendered based on the expression information. A user interface is output for presentation which requests user authorization for the client application to access the expression information. In response to receiving user input indicating user authorization, enabling access to the expression information is enabled. The client application obtains periodic updates to the expression information, and the avatar is rendered based on the periodic updates.

Abstract: Wearable systems for privacy preserving expression generation for augmented or virtual reality client applications. An example method includes receiving, by an expression manager configured to communicate expression information to client applications, a request from a client application for access to the expression information. The expression information reflects information derived from one or more sensors of the wearable system, with the client application being configured to present virtual content including an avatar rendered based on the expression information. A user interface is output for presentation which requests user authorization for the client application to access the expression information. In response to receiving user input indicating user authorization, enabling access to the expression information is enabled. The client application obtains periodic updates to the expression information, and the avatar is rendered based on the periodic updates.

Abstract: Examples of systems and methods for improved hand tracking of a user in a mixed reality environment are disclosed. The systems and methods may be configured to estimate the hand pose and shape of a user's hands for applications such as animating a hand on a user's avatar. Data from multiple sources, such as a totem internal measurement unit (“IMU”), external totem location tracking, vision cameras, and depth sensors, may be manipulated using a set of rules that are based on historical data, ergonomics data, and motion data.

Abstract: Examples of systems and methods for improved hand tracking of a user in a mixed reality environment are disclosed. The systems and methods may be configured to estimate the hand pose and shape of a user's hands for applications such as animating a hand on a user's avatar. Data from multiple sources, such as a totem internal measurement unit (“IMU”), external totem location tracking, vision cameras, and depth sensors, may be manipulated using a set of rules that are based on historical data, ergonomics data, and motion data.

Abstract: Examples of systems and methods for improved hand tracking of a user in a mixed reality environment are disclosed. The systems and methods may be configured to estimate the hand pose and shape of a user's hands for applications such as animating a hand on a user's avatar. Data from multiple sources, such as a totem internal measurement unit (“IMU”), external totem location tracking, vision cameras, and depth sensors, may be manipulated using a set of rules that are based on historical data, ergonomics data, and motion data.

Abstract: Examples of systems and methods for improved hand tracking of a user in a mixed reality environment are disclosed. The systems and methods may be configured to estimate the hand pose and shape of a user's hands for applications such as animating a hand on a user's avatar. Data from multiple sources, such as a totem internal measurement unit (“IMU”), external totem location tracking, vision cameras, and depth sensors, may be manipulated using a set of rules that are based on historical data, ergonomics data, and motion data.

Abstract: Wearable systems for privacy preserving expression generation for augmented or virtual reality client applications. An example method includes receiving, by an expression manager configured to communicate expression information to client applications, a request from a client application for access to the expression information. The expression information reflects information derived from one or more sensors of the wearable system, with the client application being configured to present virtual content including an avatar rendered based on the expression information. A user interface is output for presentation which requests user authorization for the client application to access the expression information. In response to receiving user input indicating user authorization, enabling access to the expression information is enabled. The client application obtains periodic updates to the expression information, and the avatar is rendered based on the periodic updates.

Abstract: Examples of systems and methods to facilitate audiovisual presence transitions of virtual objects such as virtual avatars in a mixed reality collaborative environment are disclosed. The systems and methods may be configured to produce different audiovisual presence transitions such as appearance, disappearance and reappearance of the virtual avatars. The virtual avatar audiovisual transitions may be further indicated by various visual and sound effects of the virtual avatars. The transitions may occur based on various colocation or decolocation scenarios.

Abstract: Examples of systems and methods for improved hand tracking of a user in a mixed reality environment are disclosed. The systems and methods may be configured to estimate the hand pose and shape of a user's hands for applications such as animating a hand on a user's avatar. Data from multiple sources, such as a totem internal measurement unit (“IMU”), external totem location tracking, vision cameras, and depth sensors, may be manipulated using a set of rules that are based on historical data, ergonomics data, and motion data.

Abstract: Various systems and methods for projecting a remote object are described herein. In one example, a method includes collecting environment data corresponding to a local environment in which a system is located and detecting a remote object corresponding to a remote user in a remote environment. The method can also include detecting a viewpoint of a local user in the local environment, and projecting the remote object corresponding to the remote user in the local environment based on the viewpoint of the local user, the virtual copy of the remote object to be positioned in the local environment by taking into account geometry of local objects in the local environment.

Abstract: Various systems and methods for projecting a remote object are described herein. In one example, a method includes collecting environment data corresponding to a local environment in which a system is located and detecting a remote object corresponding to a remote user in a remote environment. The method can also include detecting a viewpoint of a local user in the local environment, and projecting the remote object corresponding to the remote user in the local environment based on the viewpoint of the local user, the virtual copy of the remote object to be positioned in the local environment by taking into account geometry of local objects in the local environment.