Abstract: A client device provides AR content to a user tracks features of the user's ears. The client device builds a model of the user's head that represents the distances between the user's facial features and features of the user's ears. The client device generates this user head model by identifying facial feature points in an image. The client device also applies an ear feature detection model to the image, which identifies 2D points in the image where ear features are depicted. The client device generates the model based on the 3D facial feature points and the 2D ear feature points. For example, the client device may use a feature relationship model that describes the general relationships between facial features and ear points. Once the client device has generated the user head model, the client device uses the user head model to estimate the position of the ear features in 3D.

Abstract: Implementations generally relate to metaverse content modality mapping. In some implementations, a method includes obtaining functionality developed for a first modality of a virtual environment. The method further includes mapping the functionality to a second modality of the virtual environment. The method further includes executing the functionality developed for the first modality based on user interaction associated with the second modality.