Abstract: Examples in the present disclosure relate to scale estimation for facilitating extended reality (XR) experiences. An image of a hand of a user is captured via one or more optical sensors of an XR device. The image is processed to detect a hand pose relative to the XR device. A hand scale estimate corresponding to the detected hand pose is accessed. The hand scale estimate is one of a plurality of hand scale estimates each uniquely associated with a respective hand pose. The hand scale estimate is applied to generate positional data for one or more features of the hand of the user. The XR device tracks the hand of the user based on the positional data while the user uses the XR device.

Abstract: Examples in the present disclosure relate to systems and methods for detecting and rejecting a non-user hand in the context of egocentric hand tracking performed by an extended reality (XR) device. While the XR device is worn by a user, the XR device captures at least one image of a hand and processes the at least one image to detect the hand. After detecting the hand, the XR device determines positioning of the hand relative to the XR device or another object in a field of view of the XR device. The XR device detects that the hand is a non-user hand. In response to detecting that the hand is a non-user hand, the XR device excludes the non-user hand from the egocentric hand tracking such that the non-user hand is not tracked for the user.

Abstract: Devices and methods for dynamic power configuration (e.g., reduction) for thermal management (e.g., mitigation) in a wearable electronic device such as an eyewear device. The wearable electronic device monitors its temperature and, responsive to the temperature, configures the services it provides to operate in different modes for thermal mitigation (e.g., to prevent overheating). For example, based on temperature, the wearable electronic device adjusts sensors (e.g., turns cameras on or off, changes the sampling rate, or a combination thereof) and adjusts display components (e.g., adjusted rate at which a graphical processing unit generates images and a visual display is updated). This enables the wearable electronic device to consume less power when temperatures are too high in order to provide thermal mitigation.

Abstract: Examples describe a method performed by an extended reality (XR) device that implements a multi-camera object tracking system. The XR device accesses object tracking data associated with an object in a real-world environment. Based on the object tracking data, the XR device activates a low-power mode of the multi-camera object tracking system. In the low-power mode, a state of the object in the real-world environment is determined by using the multi-camera object tracking system.

Abstract: Examples in the present disclosure relate to systems and methods for reducing noise in object tracking data. Images of an object are obtained via one or more cameras. The images are processed to obtain first pose data indicative of a pose of the object over time. The first pose data is represented in a camera space. The first pose data is transformed to second pose data represented in a world space. The second pose data is filtered using a smoothing filter to generate filtered pose data. The filtering includes, for each pose data item in a time series of the second pose data, using a rotation transformation between the world space and camera space to apply one or more camera space-specific filter parameters to the pose data item that is represented in the world space. The pose of the object is dynamically tracked based on the filtered pose data.

Abstract: Examples in the present disclosure relate to systems and methods for detecting and rejecting a non-user hand in the context of egocentric hand tracking performed by an extended reality (XR) device. While the XR device is worn by a user, the XR device captures at least one image of a hand and processes the at least one image to detect the hand. After detecting the hand, the XR device determines positioning of the hand relative to the XR device or another object in a field of view of the XR device. The XR device detects that the hand is a non-user hand. In response to detecting that the hand is a non-user hand, the XR device excludes the non-user hand from the egocentric hand tracking such that the non-user hand is not tracked for the user.

Abstract: Examples in the present disclosure relate to systems and methods for detecting and rejecting a non-user hand in the context of egocentric hand tracking performed by an extended reality (XR) device. While the XR device is worn by a user, the XR device captures at least one image of a hand and processes the at least one image to detect the hand. After detecting the hand, the XR device determines positioning of the hand relative to the XR device or another object in a field of view of the XR device. The XR device detects that the hand is a non-user hand. In response to detecting that the hand is a non-user hand, the XR device excludes the non-user hand from the egocentric hand tracking such that the non-user hand is not tracked for the user.

Abstract: Examples in the present disclosure relate to the prediction of motion of a body part by an extended reality (XR) device. Tracking data is captured by one or more sensors associated with the XR device. The tracking data is processed to track the body part. Based on the tracking of the body part and a kinematic model of the body part, kinematic state tracking data is dynamically updated. The kinematic model and the kinematic state tracking data are used to generate a predicted future kinematic state of the body part. In some examples, operation of the XR device is controlled based on the predicted future kinematic state.

Abstract: In examples described herein, a sensor external to an extended reality (XR) device is connected to an extremity of a user of the XR device. The external sensor is communicatively coupled to the XR device. The XR device captures image data comprising one or more images of the extremity of the user. The XR device accesses external tracking data generated by the external sensor. A forecast of a pose of the extremity is generated based on the image data and the external tracking data. The forecast may be used for tracking of the extremity or to render virtual content for presentation to the user.

Abstract: Examples in the present disclosure relate to hand chirality estimation. Tracking data captured by one or more sensors associated with an extended reality (XR) device is processed to determine positions of a plurality of joints of a hand of a person. A reference vector is generated based on a first subset of the positions. The first subset of the positions includes positions of at least two metacarpophalangeal joints. A plurality of bending angles is determined based on at least a second subset of the positions. Each bending angle represents an angle between a respective pair of articulating bones that is measured in relation to the reference vector. An estimated chirality of the hand is identified based on the plurality of bending angles. Operation of the XR device is controlled using the estimated chirality of the hand.

Abstract: Examples disclosed herein relate to the use of shared pose data in extended reality (XR) tracking. A communication link is established between a first XR device and a second XR device. The second XR device is worn by a user. The first XR device receives pose data of the second XR device via the communication link and captures an image of the user. The user is identified based on the image and the pose data.

Abstract: Devices and methods for dynamic power configuration (e.g., reduction) for thermal management (e.g., mitigation) in a wearable electronic device such as an eyewear device. The wearable electronic device monitors its temperature and, responsive to the temperature, configures the services it provides to operate in different modes for thermal mitigation (e.g., to prevent overheating). For example, based on temperature, the wearable electronic device adjusts sensors (e.g., turns cameras on or off, changes the sampling rate, or a combination thereof) and adjusts display components (e.g., adjusted rate at which a graphical processing unit generates images and a visual display is updated). This enables the wearable electronic device to consume less power when temperatures are too high in order to provide thermal mitigation.

Abstract: Examples in the present disclosure relate to hand chirality estimation. Tracking data captured by one or more sensors associated with an extended reality (XR) device is processed to determine positions of a plurality of joints of a hand of a person. A reference vector is generated based on a first subset of the positions. The first subset of the positions includes positions of at least two metacarpophalangeal joints. A plurality of bending angles is determined based on at least a second subset of the positions. Each bending angle represents an angle between a respective pair of articulating bones that is measured in relation to the reference vector. An estimated chirality of the hand is identified based on the plurality of bending angles. Operation of the XR device is controlled using the estimated chirality of the hand.

Abstract: Examples in the present disclosure relate to the prediction of motion of a body part by an extended reality (XR) device. Tracking data is captured by one or more sensors associated with the XR device. The tracking data is processed to track the body part. Based on the tracking of the body part and a kinematic model of the body part, kinematic state tracking data is dynamically updated. The kinematic model and the kinematic state tracking data are used to generate a predicted future kinematic state of the body part. In some examples, operation of the XR device is controlled based on the predicted future kinematic state.

Abstract: In examples described herein, a sensor external to an extended reality (XR) device is connected to an extremity of a user of the XR device. The external sensor is communicatively coupled to the XR device. The XR device captures image data comprising one or more images of the extremity of the user. The XR device accesses external tracking data generated by the external sensor. A forecast of a pose of the extremity is generated based on the image data and the external tracking data. The forecast may be used for tracking of the extremity or to render virtual content for presentation to the user.

Abstract: Examples in the present disclosure relate to hand chirality estimation. Tracking data captured by one or more sensors associated with an extended reality (XR) device is processed to determine positions of a plurality of joints of a hand of a person. A reference vector is generated based on a first subset of the positions. The first subset of the positions includes positions of at least two metacarpophalangeal joints. A plurality of bending angles is determined based on at least a second subset of the positions. Each bending angle represents an angle between a respective pair of articulating bones that is measured in relation to the reference vector. An estimated chirality of the hand is identified based on the plurality of bending angles. Operation of the XR device is controlled using the estimated chirality of the hand.

Abstract: Examples disclosed herein relate to the use of shared pose data in extended reality (XR) tracking. A communication link is established between a first XR device and a second XR device. The second XR device is worn by a user. The first XR device receives pose data of the second XR device via the communication link and captures an image of the user. The user is identified based on the image and the pose data.

Abstract: A hand-tracking input pipeline dimming system for an AR system is provided. The AR system deactivates the hand-tracking input pipeline and places a camera component of the hand-tracking input pipeline in a limited operational mode. The AR system uses the camera component to detect initiation of a gesture by a user of the AR system and in response to detecting the initiation of the gesture, the AR system activates the hand-tracking input pipeline and places the camera component in a fully operational mode.

Abstract: Visual-inertial tracking of an eyewear device using a rolling shutter camera(s). The device includes a position determining system. Visual-inertial tracking is implemented by sensing motion of the device. An initial pose is obtained for a rolling shutter camera and an image of an environment is captured. The image includes feature points captured at a particular capture time. A number of poses for the rolling shutter camera is computed based on the initial pose and sensed movement of the device. The number of computed poses is responsive to the sensed movement of the mobile device. A computed pose is selected for each feature point in the image by matching the particular capture time for the feature point to the particular computed time for the computed pose. The position of the mobile device is determined within the environment using the feature points and the selected computed poses for the feature points.

Abstract: A method for limiting motion blur in a visual tracking system is described. In one aspect, the method includes accessing a first image generated by an optical sensor of the visual tracking system, identifying camera operating parameters of the optical sensor for the first image, determining a motion of the optical sensor for the first image, determining a motion blur level of the first image based on the camera operating parameters of the optical sensor and the motion of the optical sensor, and adjusting the camera operating parameters of the optical sensor based on the motion blur level.