Abstract: A forecasting system is designed to train and utilize dynamic models for forecasting metrics associated with content provision. For example, a forecasting system may identify characteristics of received training data relating to a particular title. The characteristics may be used to select one of a plurality of available forecasting models to train.

Abstract: Systems, apparatuses, methods, and computer program products are disclosed for unsupervised named entity recognition. An example method includes receiving, by a communications circuitry, a reference named entity list, the reference named entity list identifying a set of named entities and an entity type of each identified named entity. The example method further includes generating, by a vectorizer, vectors from the named entities identified in the reference named entity list, and consolidating, by a synthesizer, the generated vectors into a set of representative vectors, wherein each representative vector is associated with a particular entity type. Finally, the example method receiving, by an analysis engine, a set of text, and performing, by the analysis engine, named entity recognition on the set of text using the set of representative vectors to generate a tagged set of text.

Abstract: In general, this disclosure describes techniques for generating and evaluating automatic transcripts of audio recordings containing human speech. In some examples, a computing system is configured to: generate transcripts of a plurality of audio recordings; determine an error rate for each transcript by comparing the transcript to a reference transcript of the audio recording; receive, for each transcript, a subjective ranking selected from a plurality of subjective rank categories; determine, based on the error rates and subjective rankings, objective rank categories defined by error-rate ranges; and assign an objective ranking to a new machine-generated transcript of a new audio recording, based on the objective rank categories and an error rate of the new machine-generated transcript.

Abstract: In general, this disclosure describes techniques for generating and evaluating automatic transcripts of audio recordings containing human speech. In some examples, a computing system is configured to: generate transcripts of a plurality of audio recordings; determine an error rate for each transcript by comparing the transcript to a reference transcript of the audio recording; receive, for each transcript, a subjective ranking selected from a plurality of subjective rank categories; determine, based on the error rates and subjective rankings, objective rank categories defined by error-rate ranges; and assign an objective ranking to a new machine-generated transcript of a new audio recording, based on the objective rank categories and an error rate of the new machine-generated transcript.

Abstract: Systems and methods for detecting demographic bias in automatic speech recognition (ASR) systems. Corpuses of transcriptions from different demographic groups are analyzed, where one of the groups is known to be susceptible to bias and another group is known not to be susceptible to bias. A difference between the transcription accuracy for the first group and a transcription accuracy for a second group is measured. ASR accuracy for each group is measured and compared to each other using both statistics-based and practicality-based methodologies to determine whether a given ASR system or model exhibits a meaningful level of bias. Based on the statistical significance and the practical significance, an alert including a recommendation to adjust the ASR model is generated.

Abstract: Systems and methods for detecting demographic bias in automatic speech recognition (ASR) systems. Corpuses of transcriptions from different demographic groups are analyzed, where one of the groups is known to be susceptible to bias and another group is known not to be susceptible to bias. ASR accuracy for each group is measured and compared to each other using both statistics-based and practicality-based methodologies to determine whether a given ASR system or model exhibits a meaningful level of bias.