Abstract: Hand tracking is performed using a head-worn device and a wearable accessory device. The head-worn device is equipped with an outward-facing camera and captures images of an environment. When the wearable accessory device is in the field of view of the camera, hand tracking can be initialized by establishing a common reference between the head-worn device and the wearable accessory device. The hand is then tracked using non-image data captured by the wearable accessory device. A neural pose network processes the non-image data to predict hand pose information. Occasionally, the camera of the head-worn device is powered on, and additional image data is captured and used to perform drift correction.

Abstract: In some implementations, techniques may include at a plurality of times while a user of the first portable device is moving with the first portable device: performing ranging at a respective position with a second device to determine a respective distance, thereby determining a plurality of respective distances, where the second device is stationary; obtaining raw measurements from a motion sensor of the first portable device. In addition, the device may include using the raw measurements at the plurality of times to determine relative positions at the plurality of times, the relative position from an initial position. The techniques may include estimating a second position of the second device that optimizes a loss function that includes differences of the respective distances at the relative positions and the actual distance between the relative positions and the second position.