Abstract: Examples in the present disclosure relate to the prediction of motion of a body part by an extended reality (XR) device. Tracking data is captured by one or more sensors associated with the XR device. The tracking data is processed to track the body part. Based on the tracking of the body part and a kinematic model of the body part, kinematic state tracking data is dynamically updated. The kinematic model and the kinematic state tracking data are used to generate a predicted future kinematic state of the body part. In some examples, operation of the XR device is controlled based on the predicted future kinematic state.

Abstract: Examples describe a method performed by an extended reality (XR) device that implements a multi-camera object tracking system. The XR device accesses object tracking data associated with an object in a real-world environment. Based on the object tracking data, the XR device activates a low-power mode of the multi-camera object tracking system. In the low-power mode, a state of the object in the real-world environment is determined by using the multi-camera object tracking system.

Abstract: Examples in the present disclosure relate to systems and methods for reducing noise in object tracking data. Images of an object are obtained via one or more cameras. The images are processed to obtain first pose data indicative of a pose of the object over time. The first pose data is represented in a camera space. The first pose data is transformed to second pose data represented in a world space. The second pose data is filtered using a smoothing filter to generate filtered pose data. The filtering includes, for each pose data item in a time series of the second pose data, using a rotation transformation between the world space and camera space to apply one or more camera space-specific filter parameters to the pose data item that is represented in the world space. The pose of the object is dynamically tracked based on the filtered pose data.

Abstract: An eXtended Reality (XR) system that determines a non-interaction intent of a user is provided. The XR system captures hand tracking data using tracking sensors that include cameras capable of capturing hand movements and gestures in real-time. The XR system also captures pose data of the head-wearable apparatus using pose sensors including an Inertial Measurement Unit (IMU) and cameras to determine Six Degrees of Freedom (6 DoF) data. The XR system detects non-interaction indicators by analyzing the hand tracking data to identify situations where the user likely does not intend to interact with virtual content. Based on these detected non-interaction indicators, the system modifies virtual interaction capabilities of an XR user interface such as by selectively enabling or disabling virtual cursor feedback while maintaining direct manipulation abilities.

Abstract: Examples in the present disclosure relate to scale estimation for facilitating extended reality (XR) experiences. An image of a hand of a user is captured via one or more optical sensors of an XR device. The image is processed to detect a hand pose relative to the XR device. A hand scale estimate corresponding to the detected hand pose is accessed. The hand scale estimate is one of a plurality of hand scale estimates each uniquely associated with a respective hand pose. The hand scale estimate is applied to generate positional data for one or more features of the hand of the user. The XR device tracks the hand of the user based on the positional data while the user uses the XR device.

Abstract: Examples in the present disclosure relate to systems and methods for detecting and rejecting a non-user hand in the context of egocentric hand tracking performed by an extended reality (XR) device. While the XR device is worn by a user, the XR device captures at least one image of a hand and processes the at least one image to detect the hand. After detecting the hand, the XR device determines positioning of the hand relative to the XR device or another object in a field of view of the XR device. The XR device detects that the hand is a non-user hand. In response to detecting that the hand is a non-user hand, the XR device excludes the non-user hand from the egocentric hand tracking such that the non-user hand is not tracked for the user.

Abstract: Devices and methods for dynamic power configuration (e.g., reduction) for thermal management (e.g., mitigation) in a wearable electronic device such as an eyewear device. The wearable electronic device monitors its temperature and, responsive to the temperature, configures the services it provides to operate in different modes for thermal mitigation (e.g., to prevent overheating). For example, based on temperature, the wearable electronic device adjusts sensors (e.g., turns cameras on or off, changes the sampling rate, or a combination thereof) and adjusts display components (e.g., adjusted rate at which a graphical processing unit generates images and a visual display is updated). This enables the wearable electronic device to consume less power when temperatures are too high in order to provide thermal mitigation.

Abstract: Examples describe a method performed by an extended reality (XR) device that implements a multi-camera object tracking system. The XR device accesses object tracking data associated with an object in a real-world environment. Based on the object tracking data, the XR device activates a low-power mode of the multi-camera object tracking system. In the low-power mode, a state of the object in the real-world environment is determined by using the multi-camera object tracking system.

Abstract: Examples in the present disclosure relate to systems and methods for reducing noise in object tracking data. Images of an object are obtained via one or more cameras. The images are processed to obtain first pose data indicative of a pose of the object over time. The first pose data is represented in a camera space. The first pose data is transformed to second pose data represented in a world space. The second pose data is filtered using a smoothing filter to generate filtered pose data. The filtering includes, for each pose data item in a time series of the second pose data, using a rotation transformation between the world space and camera space to apply one or more camera space-specific filter parameters to the pose data item that is represented in the world space. The pose of the object is dynamically tracked based on the filtered pose data.

Abstract: Examples in the present disclosure relate to systems and methods for detecting and rejecting a non-user hand in the context of egocentric hand tracking performed by an extended reality (XR) device. While the XR device is worn by a user, the XR device captures at least one image of a hand and processes the at least one image to detect the hand. After detecting the hand, the XR device determines positioning of the hand relative to the XR device or another object in a field of view of the XR device. The XR device detects that the hand is a non-user hand. In response to detecting that the hand is a non-user hand, the XR device excludes the non-user hand from the egocentric hand tracking such that the non-user hand is not tracked for the user.

Abstract: Examples in the present disclosure relate to systems and methods for detecting and rejecting a non-user hand in the context of egocentric hand tracking performed by an extended reality (XR) device. While the XR device is worn by a user, the XR device captures at least one image of a hand and processes the at least one image to detect the hand. After detecting the hand, the XR device determines positioning of the hand relative to the XR device or another object in a field of view of the XR device. The XR device detects that the hand is a non-user hand. In response to detecting that the hand is a non-user hand, the XR device excludes the non-user hand from the egocentric hand tracking such that the non-user hand is not tracked for the user.

Abstract: Examples in the present disclosure relate to the prediction of motion of a body part by an extended reality (XR) device. Tracking data is captured by one or more sensors associated with the XR device. The tracking data is processed to track the body part. Based on the tracking of the body part and a kinematic model of the body part, kinematic state tracking data is dynamically updated. The kinematic model and the kinematic state tracking data are used to generate a predicted future kinematic state of the body part. In some examples, operation of the XR device is controlled based on the predicted future kinematic state.

Abstract: In examples described herein, a sensor external to an extended reality (XR) device is connected to an extremity of a user of the XR device. The external sensor is communicatively coupled to the XR device. The XR device captures image data comprising one or more images of the extremity of the user. The XR device accesses external tracking data generated by the external sensor. A forecast of a pose of the extremity is generated based on the image data and the external tracking data. The forecast may be used for tracking of the extremity or to render virtual content for presentation to the user.

Abstract: Examples in the present disclosure relate to hand chirality estimation. Tracking data captured by one or more sensors associated with an extended reality (XR) device is processed to determine positions of a plurality of joints of a hand of a person. A reference vector is generated based on a first subset of the positions. The first subset of the positions includes positions of at least two metacarpophalangeal joints. A plurality of bending angles is determined based on at least a second subset of the positions. Each bending angle represents an angle between a respective pair of articulating bones that is measured in relation to the reference vector. An estimated chirality of the hand is identified based on the plurality of bending angles. Operation of the XR device is controlled using the estimated chirality of the hand.

Abstract: Examples disclosed herein relate to the use of shared pose data in extended reality (XR) tracking. A communication link is established between a first XR device and a second XR device. The second XR device is worn by a user. The first XR device receives pose data of the second XR device via the communication link and captures an image of the user. The user is identified based on the image and the pose data.

Abstract: Devices and methods for dynamic power configuration (e.g., reduction) for thermal management (e.g., mitigation) in a wearable electronic device such as an eyewear device. The wearable electronic device monitors its temperature and, responsive to the temperature, configures the services it provides to operate in different modes for thermal mitigation (e.g., to prevent overheating). For example, based on temperature, the wearable electronic device adjusts sensors (e.g., turns cameras on or off, changes the sampling rate, or a combination thereof) and adjusts display components (e.g., adjusted rate at which a graphical processing unit generates images and a visual display is updated). This enables the wearable electronic device to consume less power when temperatures are too high in order to provide thermal mitigation.

Abstract: Examples in the present disclosure relate to the prediction of motion of a body part by an extended reality (XR) device. Tracking data is captured by one or more sensors associated with the XR device. The tracking data is processed to track the body part. Based on the tracking of the body part and a kinematic model of the body part, kinematic state tracking data is dynamically updated. The kinematic model and the kinematic state tracking data are used to generate a predicted future kinematic state of the body part. In some examples, operation of the XR device is controlled based on the predicted future kinematic state.

Abstract: Examples in the present disclosure relate to hand chirality estimation. Tracking data captured by one or more sensors associated with an extended reality (XR) device is processed to determine positions of a plurality of joints of a hand of a person. A reference vector is generated based on a first subset of the positions. The first subset of the positions includes positions of at least two metacarpophalangeal joints. A plurality of bending angles is determined based on at least a second subset of the positions. Each bending angle represents an angle between a respective pair of articulating bones that is measured in relation to the reference vector. An estimated chirality of the hand is identified based on the plurality of bending angles. Operation of the XR device is controlled using the estimated chirality of the hand.

Abstract: Examples in the present disclosure relate to hand chirality estimation. Tracking data captured by one or more sensors associated with an extended reality (XR) device is processed to determine positions of a plurality of joints of a hand of a person. A reference vector is generated based on a first subset of the positions. The first subset of the positions includes positions of at least two metacarpophalangeal joints. A plurality of bending angles is determined based on at least a second subset of the positions. Each bending angle represents an angle between a respective pair of articulating bones that is measured in relation to the reference vector. An estimated chirality of the hand is identified based on the plurality of bending angles. Operation of the XR device is controlled using the estimated chirality of the hand.

Abstract: In examples described herein, a sensor external to an extended reality (XR) device is connected to an extremity of a user of the XR device. The external sensor is communicatively coupled to the XR device. The XR device captures image data comprising one or more images of the extremity of the user. The XR device accesses external tracking data generated by the external sensor. A forecast of a pose of the extremity is generated based on the image data and the external tracking data. The forecast may be used for tracking of the extremity or to render virtual content for presentation to the user.