Abstract: The present disclosure may be embodied as systems and methods for action recognition developed using a multimodal dataset that incorporates both visual data, which facilitates the accurate tracking of movement, and active acoustic data, which captures the micro-Doppler modulations induced by the motion. The dataset includes twenty-one actions and focuses on examples of orientational symmetry that a single active ultrasound sensor should have the most difficulty discriminating. The combined results from three independent ultrasound sensors are encouraging, and provide a foundation to explore the use of data from multiple viewpoints to resolve the orientational ambiguity in action recognition. In various embodiments, recurrent neural networks using long short-term memory (LSTM) or hidden Markov models (HMMs) are disclosed for use in action recognition, for example, human action recognition, from micro-Doppler signatures.

Abstract: The present disclosure may be embodied as systems and methods for action recognition developed using a multimodal dataset that incorporates both visual data, which facilitates the accurate tracking of movement, and active acoustic data, which captures the micro-Doppler modulations induced by the motion. The dataset includes twenty-one actions and focuses on examples of orientational symmetry that a single active ultrasound sensor should have the most difficulty discriminating. The combined results from three independent ultrasound sensors are encouraging, and provide a foundation to explore the use of data from multiple viewpoints to resolve the orientational ambiguity in action recognition. In various embodiments, recurrent neural networks using long short-term memory (LSTM) or hidden Markov models (HMMs) are disclosed for use in action recognition, for example, human action recognition, from micro-Doppler signatures.