Abstract: An automated testing system and method for evaluating voice processing systems is provided. In one embodiment, a method includes receiving a plurality of voice command inputs and a plurality of expected responses associated with the voice command inputs. A text-to-speech engine is applied to the voice command inputs to generate test command audio files. The test command audio files are provided to a testing apparatus in communication with a voice processing system. A generated response output from the voice processing system is obtained for each of the test command audio files. The generated response is captured from the testing apparatus using a sensor to detect audio and/or visual information. The obtained generated response is compared to an expected response from the plurality of expected responses for each of the test command audio files. Based on the comparison, a test result is provided for each of the voice command inputs.

Abstract: An automated testing system and method for evaluating voice processing systems is provided. In one embodiment, a method includes receiving a plurality of voice command inputs and a plurality of expected responses associated with the voice command inputs. A text-to-speech engine is applied to the voice command inputs to generate test command audio files. The test command audio files are provided to a testing apparatus in communication with a voice processing system. A generated response output from the voice processing system is obtained for each of the test command audio files. The generated response is captured from the testing apparatus using a sensor to detect audio and/or visual information. The obtained generated response is compared to an expected response from the plurality of expected responses for each of the test command audio files. Based on the comparison, a test result is provided for each of the voice command inputs.