Abstract: This disclosure relates generally to ASR and is particularly directed to automatic, efficient, and intelligent detection of transcription bias in ASR models. Contrary to a tradition approach to the testing of ASR bias, the example implementations disclosed herein do not require actual test speeches and corresponding ground-truth texts. Instead, test speeches may be machine-generated from a pre-constructed reference textual passage according short speech samples of speakers using a neural voice cloning technology. The reference passage may be constructed according to a particular target domain of the ASR model being tested. Bias of the ASR model in various aspects may be identified by analyzing transcribed text from the machine-generated speeches and the reference textual passage. The underlying principles for bias detection may be applied to evaluation of general transcription effectiveness and accuracy of the ASR model.

Abstract: This disclosure relates generally to ASR and is particularly directed to automatic, efficient, and intelligent detection of transcription bias in ASR models. Contrary to a tradition approach to the testing of ASR bias, the example implementations disclosed herein do not require actual test speeches and corresponding ground-truth texts. Instead, test speeches may be machine-generated from a pre-constructed reference textual passage according short speech samples of speakers using a neural voice cloning technology. The reference passage may be constructed according to a particular target domain of the ASR model being tested. Bias of the ASR model in various aspects may be identified by analyzing transcribed text from the machine-generated speeches and the reference textual passage. The underlying principles for bias detection may be applied to evaluation of general transcription effectiveness and accuracy of the ASR model.