Abstract: An XR device is provided that determines a proximity of a hand of a user to the XR device using tracking data and detects a vibration indicating physical contact with the XR device using a vibration detection sensor. The XR device activates at least one audio sensor in response to the detected vibration and the proximity, analyzes audio data captured by the audio sensor to determine a touch gesture type, and performs a function based on the determined touch gesture type. The XR device uses multi-modal sensor fusion to detect touch gestures without requiring physical buttons, balancing informativeness, power efficiency, and privacy by selectively activating sensors based on their characteristics.

Abstract: An eXtended Reality (XR) device is provided that uses images of a dorsal surface of a hand of a user to authenticate the user. The XR device captures, using a set of cameras, images of a hand of a user and processes these images to generate enhanced vein patterns. The XR device generates authentication features based on the enhanced vein patterns and performs classification using the authentication features. Based on the classification, the XR device controls access to specific content and applications. The XR device includes an infrared camera and emitter to optimize vein pattern visibility, processes images using computer vision algorithms, and performs lightweight classification on-device while maintaining user privacy.

Abstract: A system comprising a head-wearable apparatus, which includes a plurality of microphones, each microphone of the plurality of microphones being spatially separated from each other microphone of the plurality of microphones, thereby defining a set of spatial separations. The system also includes one or more processors. The system also includes a non-transitory computer readable storage medium including instructions that, when executed by the one or more processors, cause the one or more processors to perform operations including generating, by each microphone of the plurality of microphones, an audio signal corresponding to sound detected by the microphone, thereby generating a plurality of audio signals, and processing the plurality of audio signals, using a trained machine learning model, to generate wearer speech detection data representative of a likelihood that a wearer of the head-wearable apparatus is speaking.