Abstract: The subject technology receives a set of inputs from multiple input sources. The subject technology determines a set of input features based on the set of inputs from the multiple input sources. The subject technology performs a time window-based aggregation on the set of input features to generate a set of aggregated features. The subject technology performs feature extraction, using a set of modular components of a modular classifier network, on the set of aggregated features to generate a set of extracted features. The subject technology generates, using a pinch detection head, a probability score indicating the likelihood of an occurrence of a pinch gesture based on the set of extracted features. The subject technology determines, using triggering logic, whether a pinch gesture has occurred based at least in part on the probability score. The subject technology provides a pinch detection output based at least in part on the determining.

Abstract: A hand-tracking platform generates gesture components for use as user inputs into an application of an Augmented Reality (AR) system. In some examples, the hand-tracking platform generates real-world scene environment frame data based on gestures being made by a user of the AR system using a camera component of the AR system. The hand-tracking platform recognizes a gesture component based on the real-world scene environment frame data and generates gesture component data based on the gesture component. The application utilizes the gesture component data as user input in a user interface of the application.

Abstract: The subject technology receives a set of inputs from multiple input sources. The subject technology determines a set of input features based on the set of inputs from the multiple input sources. The subject technology performs a time window-based aggregation on the set of input features to generate a set of aggregated features. The subject technology performs feature extraction, using a set of modular components of a modular classifier network, on the set of aggregated features to generate a set of extracted features. The subject technology generates, using a pinch detection head, a probability score indicating the likelihood of an occurrence of a pinch gesture based on the set of extracted features. The subject technology determines, using triggering logic, whether a pinch gesture has occurred based at least in part on the probability score. The subject technology provides a pinch detection output based at least in part on the determining.

Abstract: An extended Reality (XR) system is provided that provides filtering for a raycast cursor. The XR system provides an XR user interface to a user, the XR user interface includes a raycast and pinch user input modality having a raycast cursor. While continuously capturing, using one or more sensors of the XR system, tracking data of a hand of a user, the XR system determines a pinch intention intensity using the tracking data, and in response to determining the pinch intention intensity meets or exceeds an enter pinching state value, activates a raycast cursor filter applied to a position of the raycast cursor of the raycast and pinch user input modality.

Abstract: A head-worn device system includes multiple image sensors (e.g., cameras), one or more display devices and one or more processors. The system also includes a memory storing instructions that, when executed by the one or more processors, configure the system to obtain a first image captured by a first image sensor of the device; generate, based on obtaining the first image, a respective predicted skeleton corresponding to a respective view from each of the first image sensor and the one or more second image sensors; select, based on generating the respective predicted skeletons, an image sensor from among the first image sensor and the one or more second image sensors, the selected image sensor being used for classifying the first image; and determine, based on the selected image sensor, a classification for the first image.

Abstract: The subject technology receives a set of inputs from multiple input sources. The subject technology determines a set of input features based on the set of inputs from the multiple input sources. The subject technology performs a time window-based aggregation on the set of input features to generate a set of aggregated features. The subject technology performs feature extraction, using a set of modular components of a modular classifier network, on the set of aggregated features to generate a set of extracted features. The subject technology generates, using a pinch detection head, a probability score indicating the likelihood of an occurrence of a pinch gesture based on the set of extracted features. The subject technology determines, using triggering logic, whether a pinch gesture has occurred based at least in part on the probability score. The subject technology provides a pinch detection output based at least in part on the determining.

Abstract: A hand-tracking platform generates gesture components for use as user inputs into an application of an Augmented Reality (AR) system. In some examples, the hand-tracking platform generates real-world scene environment frame data based on gestures being made by a user of the AR system using a camera component of the AR system. The hand-tracking platform recognizes a gesture component based on the real-world scene environment frame data and generates gesture component data based on the gesture component. The application utilizes the gesture component data as user input in a user interface of the application.

Abstract: A hand-tracking platform generates gesture components for use as user inputs into an application of an Augmented Reality (AR) system. In some examples, the hand-tracking platform generates real-world scene environment frame data based on gestures being made by a user of the AR system using a camera component of the AR system. The hand-tracking platform recognizes a gesture component based on the real-world scene environment frame data and generates gesture component data based on the gesture component. The application utilizes the gesture component data as user input in a user interface of the application.

Abstract: A system and method for translating sign language utterances into a target language, including: receiving motion capture data; producing phonemes/sign fragments from the received motion capture data; producing a plurality of sign sequences from the phonemes/sign fragments; parsing these sign sequences to produce grammatically parsed sign utterances; translating the grammatically parsed sign utterances into grammatical representations in the target language; and generating output utterances in the target language based upon the grammatical representations.

Abstract: A system and method for translating sign language utterances into a target language, including: receiving motion capture data; producing phonemes/sign fragments from the received motion capture data; producing a plurality of sign sequences from the phonemes/sign fragments; parsing these sign sequences to produce grammatically parsed sign utterances; translating the grammatically parsed sign utterances into grammatical representations in the target language; and generating output utterances in the target language based upon the grammatical representations.