Abstract: Methods, systems, devices, and computer-readable storage media for activity detection of a user of a computing device, using multi-modal sensing. A device can be configured to receive sensor data corresponding to multiple modalities and process the sensor data to predict an activity performed by a user of a computing device. The device in response to the detected activity can perform a response action, such as muting or pausing audio playback from the computing device. Different modalities can be combined, such as body vibration data, air vibration data, and image data, which can be processed to distinguish user activity, e.g., speaking versus not speaking, to allow the computing device to perform the correct corresponding action.

Abstract: Aspects of the disclosure relate to voice activity detection (VAD) on wearable and other resource-constrained devices, to classify speech recorded by a microphone of the device as belonging to a wearer of the device versus another speech source. A computing device can include a microphone and an inertial measurement unit (IMU). The wearable device can use signals measured by the IMU for providing motion-tracking features, such as head tracking for augmented reality or virtual reality applications. Aspects of the disclosure provide for a device for leveraging existing data collected for these motion-tracking features for use in VAD. A device can pre-process data streamed from an IMU to use only signals predetermined to be indicative of whether or not a wearer of the device is speaking.

Abstract: Methods, systems, devices, and computer-readable storage media for activity detection of a user of a computing device, using multi-modal sensing. A device can be configured to receive sensor data corresponding to multiple modalities and process the sensor data to predict an activity performed by a user of a computing device. The device in response to the detected activity can perform a response action, such as muting or pausing audio playback from the computing device. Different modalities can be combined, such as body vibration data, air vibration data, and image data, which can be processed to distinguish user activity, e.g., speaking versus not speaking, to allow the computing device to perform the correct corresponding action.

Abstract: Methods, systems, devices, and computer-readable storage media for activity detection of a user of a computing device, using multi-modal sensing. A device can be configured to receive sensor data corresponding to multiple modalities and process the sensor data to predict an activity performed by a user of a computing device. The device in response to the detected activity can perform a response action, such as muting or pausing audio playback from the computing device. Different modalities can be combined, such as body vibration data, air vibration data, and image data, which can be processed to distinguish user activity, e.g., speaking versus not speaking, to allow the computing device to perform the correct corresponding action.

Abstract: Methods, systems, devices, and computer-readable storage media for activity detection of a user of a computing device, using multi-modal sensing. A device can be configured to receive sensor data corresponding to multiple modalities and process the sensor data to predict an activity performed by a user of a computing device. The device in response to the detected activity can perform a response action, such as muting or pausing audio playback from the computing device. Different modalities can be combined, such as body vibration data, air vibration data, and image data, which can be processed to distinguish user activity, e.g., speaking versus not speaking, to allow the computing device to perform the correct corresponding action.