Abstract: The various implementations described herein include methods and systems for using hand gestures detected at a wearable device to navigate a user interface. An example method includes receiving, via one or more neuromuscular-signal sensors of a wrist-wearable device worn by a user, data generated during performance of an in-air wrist movement by a wrist of the user. The method also includes moving a point of focus on the user interface in accordance with the in-air wrist movement; and receiving, via the one or more neuromuscular-signal sensors, additional data generated during performance of an in-air gesture by the user. The method further includes determining that the in-air gesture is an execution gesture; and executing a command corresponding to the execution gesture.

Abstract: The various implementations described herein include methods and systems for using a multi-stage in-air hand gesture to activate user-interface interactions in a way that ensures low-false positive rates. In one aspect, a method includes, while a gating gesture is maintained, receiving a first indication of performance of an adjustment gesture of a first magnitude directed to a user interface object associated with a plurality of values. The method further includes, in response to receiving the first indication, adjusting the user interface object to have a first state after moving through some of the plurality of values based on the first magnitude. The method also includes, after receiving an indication of a release of the gating gesture, in response to receiving a second indication of performance of the adjustment gesture, forgoing adjusting the user interface object such that the user interface object continues to have the first state.

Abstract: The various implementations described herein include methods and systems for producing and modifying text using neuromuscular-signal-sensing devices. In one aspect, a method includes causing the display of a plurality of text terms input by a user. Using data from one or more neuromuscular-signal sensors in communication with the wearable device, an in-air hand gesture performed by the user is detected while the text terms are displayed. In response to the in-air hand gesture, a text-modification mode is enabled that allows for modifying the text terms input by the user. A target term is identified and, while the text-modification mode is enabled, data about a voice input provided by the user for modifying the target term is received. The method further includes causing a modification to the target term in accordance with the voice input from the user.

Abstract: Implementations of the present disclosure include actions of receiving image data of an image capturing a scene, receiving data describing one or more entities determined from the scene, the one or more entities being determined from the scene, determining one or more actions based on the one or more entities, each action being provided at least partly based on search results from searching the one or more entities, and providing instructions to display an action interface comprising one or more action elements, each action element being to induce execution of a respective action, the action interface being displayed in a viewfinder

Abstract: The disclosed system for interacting with objects in an extended reality (XR) environment generated by an XR system may include (1) neuromuscular sensors configured to sense neuromuscular signals from a wrist of a user and (2) at least one computer processor programmed to (a) determine, based at least in part on the sensed neuromuscular signals, information relating to an interaction of the user with an object in the XR environment and (b) instruct the XR system to, based on the determined information relating to the interaction of the user with the object, augment the interaction of the user with the object in the XR environment. Other embodiments of this aspect include corresponding apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.

Abstract: Methods and systems for providing input to an augmented reality system or an extended reality system based, at least in part, on neuromuscular signals. The methods and systems comprise detecting, using one or more neuromuscular sensors arranged on one or more wearable devices, neuromuscular signals from a user; determining that a computerized system is in a mode configured to provide input including text to the augmented reality system; identifying based, at least in part, on the neuromuscular signals and/or information based on the neuromuscular signals, the input, wherein the input is further identified based, at least in part, on the mode; and providing the identified input to the augmented reality system.

Abstract: Implementations of the present disclosure include actions of receiving image data of an image capturing a scene, receiving data describing one or more entities determined from the scene, the one or more entities being determined from the scene, determining one or more actions based on the one or more entities, each action being provided at least partly based on search results from searching the one or more entities, and providing instructions to display an action interface comprising one or more action elements, each action element being to induce execution of a respective action, the action interface being displayed in a viewfinder.

Abstract: The disclosed computer-implemented method may include presenting, via a user interface, a sensory cue, and receiving, from neuromuscular sensors of a wearable device, various neuromuscular signals generated by a user wearing the wearable device, where the user generates the neuromuscular signals in response to the sensory cue being presented to the user via the user interface. The method may also include interpreting the received neuromuscular signals as input commands with respect to the sensory cue provided by the user interface, such that the input commands initiate performance of specified tasks within the user interface. The method may also include performing the specified tasks within the user interface according to the interpreted input commands. Various other methods, systems, and computer-readable media are also disclosed.

Abstract: The disclosed human computer interface (HCI) system may include (1) at least one processor, (2) a plurality of sensors that detect one or more neuromuscular signals from a forearm or wrist of a user, and (3) memory that stores (A) one or more trained inferential models that determine an amount of force associated with the one or more neuromuscular signals and (B) computer-executable instructions that, when executed by the at least one processor, cause the at least one processor to (I) identify the amount of force determined by the one or more trained inferential models, (II) determine that the amount of force satisfies a threshold force value, and in accordance with the determination that the amount of force satisfies the threshold force value, (III) generate a first input command for the HCI system. Various other devices, systems, and methods are also disclosed.

Abstract: Generating a playlist may include designating a seed track in an audio library; identifying audio tracks in the audio library having constructs that are within a range of a corresponding construct of the seed track, where the constructs for the audio tracks are derived from frequency representations of the audio tracks, and the corresponding construct for the seed track is derived from a frequency representation of the seed track; and generating the playlist using at least some of the audio tracks that were identified.

Abstract: Methods and apparatus for enabling a user or third-party to select or adjust parameters of one or more statistical models used to generate a musculoskeletal representation. The method comprises providing as input to the statistical model(s), a plurality of neuromuscular signals recorded by a plurality of neuromuscular sensors during performance of at least one gesture by a user, wherein the at least one gesture is performed by the user while wearing a wearable device having the plurality of neuromuscular sensors arranged thereon, rendering at least one visual representation based on an output of the statistical model(s), and receiving user or third-party input to adjust parameters of the statistical model(s) based on the rendered at least one visual representation, the user input including a selection of a particular statistical model of the statistical model(s) and/or an adjustment of parameters associated with the particular statistical model.

Abstract: Systems and methods for using neuromuscular information to improve speech recognition. The system includes a plurality of neuromuscular sensors arranged on one or more wearable devices and configured to continuously record a plurality of neuromuscular signals from a user, at least one storage device configured to store one or more trained statistical models for determining text based on audio input and the plurality of neuromuscular signals, at least one input interface configured to receive the audio input, and at least one computer processor programmed to obtain the audio input and the plurality of neuromuscular signals, provide as input to the one or more trained statistical models, the audio input and the plurality of neuromuscular signals or signals derived from the plurality of neuromuscular signals, and determine based, at least in part, on an output of the one or more trained statistical models, the text.

Abstract: Methods and apparatus for calibrating performance of one or more statistical models used to generate a musculoskeletal representation. The method comprises controlling presentation of instructions via a user interface to instruct the user to perform the at least one gesture and updating at least one parameter of the one or more statistical models based, at least in part on a plurality of neuromuscular signals recorded by a plurality of neuromuscular sensors during performance of the at least one gesture by the user.

Abstract: The disclosed human computer interface (HCI) system may include (1) at least one processor, (2) a plurality of sensors that detect one or more neuromuscular signals from a forearm or wrist of a user, and (3) memory that stores (A) one or more trained inferential models that determine an amount of force associated with the one or more neuromuscular signals and (B) computer-executable instructions that, when executed by the at least one processor, cause the at least one processor to (I) identify the amount of force determined by the one or more trained inferential models, (II) determine that the amount of force satisfies a threshold force value, and in accordance with the determination that the amount of force satisfies the threshold force value, (III) generate a first input command for the HCI system. Various other devices, systems, and methods are also disclosed.

Abstract: Implementations of the present disclosure include actions of receiving image data of an image capturing a scene, receiving data describing one or more entities determined from the scene, the one or more entities being determined from the scene, determining one or more actions based on the one or more entities, each action being provided at least partly based on search results from searching the one or more entities, and providing instructions to display an action interface comprising one or more action elements, each action element being to induce execution of a respective action, the action interface being displayed in a viewfinder

Abstract: Method and apparatus for rendering a visual representation based on a musculoskeletal representation. The method comprises updating the musculoskeletal representation based, at least in part, on a plurality of neuromuscular signals recorded from a user, wherein the musculoskeletal representation is updated based at least in part on: position information describing a spatial relationship between two or more connected segments of the musculoskeletal representation, and force information describing a force exerted by at least one segment of the musculoskeletal representation, and rendering, via a user interface, the visual representation based on the updated musculoskeletal representation, wherein the visual representation includes a visual indication of the position information and a visual indication of the force information.

Abstract: Computerized systems, methods, kits, and computer-readable media storing code for implementing the methods are provided for interacting with a physical object in an augmented reality (AR) environment generated by an AR system. One such system includes: a plurality of neuromuscular sensors able to sense a plurality of neuromuscular signals from a user, and at least one computer processor. The neuromuscular sensors are arranged on one or more wearable devices worn by the user to sense the neuromuscular signals. The at least one computer processor is or are programmed to: determine, based at least in part, on the neuromuscular signals sensed by the neuromuscular sensors, information about an interaction of the user with the physical object in the AR environment generated by the AR system; and instruct the AR system to provide feedback based, at least in part, on the information about the interaction of the user with the physical object.

Abstract: Systems and methods for using neuromuscular information to improve speech recognition. The system includes a plurality of neuromuscular sensors, arranged on one or more wearable devices, wherein the plurality of neuromuscular sensors is configured to continuously record a plurality of neuromuscular signals from a user, at least one storage device configured to store one or more trained statistical models, and at least one computer processor programmed to provide as an input to the one or more trained statistical models, the plurality of neuromuscular signals or signals derived from the plurality of neuromuscular signals, determine based, at least in part, on an output of the one or more trained statistical models, at least one instruction for modifying an operation of a speech recognizer, and provide the at least one instruction to the speech recognizer.

Abstract: Implementations of the present disclosure include actions of receiving image data of an image capturing a scene, receiving data describing one or more entities determined from the scene, the one or more entities being determined from the scene, determining one or more actions based on the one or more entities, each action being provided at least partly based on search results from searching the one or more entities, and providing instructions to display an action interface comprising one or more action elements, each action element being to induce execution of a respective action, the action interface being displayed in a viewfinder.

Abstract: Computerized systems, methods, kits, and computer-readable media storing code for implementing the methods are provided for interacting with a physical object in an augmented reality (AR) environment generated by an AR system. One such system includes: a plurality of neuromuscular sensors able to sense a plurality of neuromuscular signals from a user, and at least one computer processor. The neuromuscular sensors are arranged on one or more wearable devices worn by the user to sense the neuromuscular signals. The at least one computer processor is or are programmed to: determine, based at least in part, on the neuromuscular signals sensed by the neuromuscular sensors, information about an interaction of the user with the physical object in the AR environment generated by the AR system; and instruct the AR system to provide feedback based, at least in part, on the information about the interaction of the user with the physical object.