Abstract: A speech recognition method includes receiving audible data and user data. The audible data includes information about an utterance by the user. The user data includes information about movements by the user. The method further includes fusing the audible data and the user data to obtain fused data and determining at least one spoken word of the utterance based on the fused data.

Abstract: An augmented reality head up display system includes an audio microphone array, a driver position tracking device, and an augmented reality head up display. The augmented reality head up display system also includes a controller in electronic communication with the audio microphone array, the driver position tracking device, and the augmented reality head up display, where the controller is programmed to: receive audio data from the audio microphone array corresponding to audio detected by the audio microphone array, classify the audio data as corresponding to an audio event occurring in an environment surrounding the vehicle, and display a graphic corresponding to the audio event on the windscreen of the vehicle using the augmented reality head up display and input from the driver position tracking device.