Abstract: Examples in the present disclosure relate to scale estimation for facilitating extended reality (XR) experiences. An image of a hand of a user is captured via one or more optical sensors of an XR device. The image is processed to detect a hand pose relative to the XR device. A hand scale estimate corresponding to the detected hand pose is accessed. The hand scale estimate is one of a plurality of hand scale estimates each uniquely associated with a respective hand pose. The hand scale estimate is applied to generate positional data for one or more features of the hand of the user. The XR device tracks the hand of the user based on the positional data while the user uses the XR device.

Abstract: Examples in the present disclosure relate to scale estimation for facilitating extended reality (XR) experiences. An image of a hand of a user is captured via one or more optical sensors of an XR device. The image is processed to detect a hand pose relative to the XR device. A hand scale estimate corresponding to the detected hand pose is accessed. The hand scale estimate is one of a plurality of hand scale estimates each uniquely associated with a respective hand pose. The hand scale estimate is applied to generate positional data for one or more features of the hand of the user. The XR device tracks the hand of the user based on the positional data while the user uses the XR device.

Abstract: Examples in the present disclosure relate to systems and methods for reducing noise in object tracking data. Images of an object are obtained via one or more cameras. The images are processed to obtain first pose data indicative of a pose of the object over time. The first pose data is represented in a camera space. The first pose data is transformed to second pose data represented in a world space. The second pose data is filtered using a smoothing filter to generate filtered pose data. The filtering includes, for each pose data item in a time series of the second pose data, using a rotation transformation between the world space and camera space to apply one or more camera space-specific filter parameters to the pose data item that is represented in the world space. The pose of the object is dynamically tracked based on the filtered pose data.

Abstract: A system and method for generating and updating a posegraph for multi-user augmented reality experiences. The system receives initial pose data from multiple client devices and generates a posegraph based on this data. When client devices come within proximity of each other, the system detects this and receives relative pose observations from the devices. The system then updates the posegraph based on these observations, assigning confidence values to improve accuracy. This approach enables efficient synchronization of spatial information across devices, allowing for seamless shared AR experiences in large-scale environments without the need for complete map sharing or pre-mapped areas.

Abstract: Examples in the present disclosure relate to the prediction of motion of a body part by an extended reality (XR) device. Tracking data is captured by one or more sensors associated with the XR device. The tracking data is processed to track the body part. Based on the tracking of the body part and a kinematic model of the body part, kinematic state tracking data is dynamically updated. The kinematic model and the kinematic state tracking data are used to generate a predicted future kinematic state of the body part. In some examples, operation of the XR device is controlled based on the predicted future kinematic state.

Abstract: Examples in the present disclosure relate to hand chirality estimation. Tracking data captured by one or more sensors associated with an extended reality (XR) device is processed to determine positions of a plurality of joints of a hand of a person. A reference vector is generated based on a first subset of the positions. The first subset of the positions includes positions of at least two metacarpophalangeal joints. A plurality of bending angles is determined based on at least a second subset of the positions. Each bending angle represents an angle between a respective pair of articulating bones that is measured in relation to the reference vector. An estimated chirality of the hand is identified based on the plurality of bending angles. Operation of the XR device is controlled using the estimated chirality of the hand.

Abstract: A method for aligning coordinate systems between devices is described. The method comprises accessing pose data from a first device that defines a first coordinate system and receiving pose data from a second device that defines a second coordinate system. The method further includes identifying a coordinate transformation that relates the first device, a hand joint of a user holding the second device, and the second device, and then aligning the first coordinate system with the second based on the pose data and the identified transformation.

Abstract: Examples in the present disclosure relate to hand chirality estimation. Tracking data captured by one or more sensors associated with an extended reality (XR) device is processed to determine positions of a plurality of joints of a hand of a person. A reference vector is generated based on a first subset of the positions. The first subset of the positions includes positions of at least two metacarpophalangeal joints. A plurality of bending angles is determined based on at least a second subset of the positions. Each bending angle represents an angle between a respective pair of articulating bones that is measured in relation to the reference vector. An estimated chirality of the hand is identified based on the plurality of bending angles. Operation of the XR device is controlled using the estimated chirality of the hand.

Abstract: Examples in the present disclosure relate to the prediction of motion of a body part by an extended reality (XR) device. Tracking data is captured by one or more sensors associated with the XR device. The tracking data is processed to track the body part. Based on the tracking of the body part and a kinematic model of the body part, kinematic state tracking data is dynamically updated. The kinematic model and the kinematic state tracking data are used to generate a predicted future kinematic state of the body part. In some examples, operation of the XR device is controlled based on the predicted future kinematic state.

Abstract: Examples in the present disclosure relate to hand chirality estimation. Tracking data captured by one or more sensors associated with an extended reality (XR) device is processed to determine positions of a plurality of joints of a hand of a person. A reference vector is generated based on a first subset of the positions. The first subset of the positions includes positions of at least two metacarpophalangeal joints. A plurality of bending angles is determined based on at least a second subset of the positions. Each bending angle represents an angle between a respective pair of articulating bones that is measured in relation to the reference vector. An estimated chirality of the hand is identified based on the plurality of bending angles. Operation of the XR device is controlled using the estimated chirality of the hand.

Abstract: A first extended reality (XR) device and a second XR device are colocated in an environment. The first XR device captures sensory data of a wearer of the second XR device. The sensory data is used to determine a time offset between a first clock of the first XR device and a second clock of the second XR device. The first clock and the second clock are synchronized based on the time offset and a shared coordinate system is established. The shared coordinate system enables alignment of virtual content that is simultaneously presented by the first XR device and the second XR device based on the synchronization of the first clock and the second clock.

Abstract: A first extended reality (XR) device and a second XR device are colocated in an environment. The first XR device captures sensory data of a wearer of the second XR device. The sensory data is used to determine a time offset between a first clock of the first XR device and a second clock of the second XR device. The first clock and the second clock are synchronized based on the time offset and a shared coordinate system is established. The shared coordinate system enables alignment of virtual content that is simultaneously presented by the first XR device and the second XR device based on the synchronization of the first clock and the second clock.

Abstract: A method for aligning coordinate systems from separate handheld devices is described. In one aspect, the method includes accessing first pose data of a first handheld device, receiving second pose data of a second handheld device, detecting, from the first handheld device, hand-tracking data of a second user holding the second handheld device, and aligning a first coordinate system of the first handheld device with a second coordinate system of the second handheld device based on the first pose data, the second pose data, and the hand-tracking data of the second user holding the second handheld device.

Abstract: A first extended reality (XR) device and a second XR device are colocated in an environment. The first XR device captures sensory data of a wearer of the second XR device. The sensory data is used to determine a time offset between a first clock of the first XR device and a second clock of the second XR device. The first clock and the second clock are synchronized based on the time offset and a shared coordinate system is established. The shared coordinate system enables alignment of virtual content that is simultaneously presented by the first XR device and the second XR device based on the synchronization of the first clock and the second clock.

Abstract: A method for aligning coordinate systems from separate handheld devices is described. In one aspect, the method includes accessing first pose data of a first handheld device, receiving second pose data of a second handheld device, detecting, from the first handheld device, hand-tracking data of a second user holding the second handheld device, and aligning a first coordinate system of the first handheld device with a second coordinate system of the second handheld device based on the first pose data, the second pose data, and the hand-tracking data of the second user holding the second handheld device.