Abstract: A system configured to determine a six-degree of freedom pose of a physical object in a physical environment and to utilize the six-degree of freedom pose as an within a virtual environment or mixed reality environment. In some cases, the system may utilize one or more cameras on a headset device to track the pose of a controller or other objects and one or more cameras on the controller itself to track the pose of the headset device or the user. In one example, the system may capture image data of a physical object having a constellation or pattern on the external source. The system may analyze the image data to identify image points associated with the constellation or pattern and to determine the pose of the object based on a location of the points in the image.

Abstract: A system configured to determine a pose of a physical object in a physical environment and to utilize the pose as an input to control or manipulate a virtual environment or mixed reality environment. In some cases, the system may be capture image data of a physical object having a constellation or pattern on the external source. The system may analyze the image data to identify points associated with the constellation or pattern and to determine the pose based on the identified points and a stored object model associated with the physical object.

Abstract: A virtual environment or mixed reality environment system including a physical tracking enhancement tag that may be identified by the system to assist with localization of the system with respect to the physical environment. The tracking enhancement tags may be encoded or coded in a high frequency and low frequency that is readable by the system to provide different types of information related the environment. For example, the encoded information may include a unique identifier of the tag and a relative direction with regards to the physical environment.

Abstract: A virtual environment or mixed reality environment system including a headset device and controller. Both the headset device and the controller are configured to access and update a shared scene map. In some cases, the headset device may implement pose tracking with respect to the headset device and the controller and the controller may also implement pose tracking with respect to the headset device and the controller.