Abstract: Provided are methods, systems, devices, apparatuses, and tangible non-transitory computer readable media for controlling avatars in an augmented reality environment. The disclosed technology can receive sensor data comprising images of a physical environment and images of a user. Based on the sensor data, an augmented reality environment comprising an avatar and based on the images of the physical environment can be generated. The avatar can comprise a three-dimensional model comprising a facial region based on the images of the user. Inputs to control the avatar within the augmented reality environment can be detected and facial states of the user can be determined based on the images. Based on the inputs and the facial states, states of the avatar can be modified. The states of the avatar can comprise a position of the avatar within the augmented reality environment and a configuration of the facial region based on the facial states.

Abstract: Provided are methods, systems, devices, apparatuses, and tangible non-transitory computer readable media for controlling avatars in an augmented reality environment. The disclosed technology can receive sensor data comprising images of a physical environment and images of a user. Based on the sensor data, an augmented reality environment comprising an avatar and based on the images of the physical environment can be generated. The avatar can comprise a three-dimensional model comprising a facial region based on the images of the user. Inputs to control the avatar within the augmented reality environment can be detected and facial states of the user can be determined based on the images. Based on the inputs and the facial states, states of the avatar can be modified. The states of the avatar can comprise a position of the avatar within the augmented reality environment and a configuration of the facial region based on the facial states.