Abstract: Systems, apparatuses, methods, and computer program products are disclosed for mitigating deterioration of modeling system performance. An example method includes detecting, by context analysis circuitry, occurrence of a triggering condition. The example method also includes determining, by context analysis circuitry and in response to detecting the occurrence of the triggering condition, a destination modeling solution based on a regime strategy for the modeling system. The example method also includes switching, by regime deployment circuitry and based on the regime strategy, from a source modeling solution to a destination modeling solution.

Abstract: An example method includes detecting, by context analysis circuitry, occurrence of a triggering condition. The example method also includes scheduling, by context analysis circuitry and based on the occurrence of the triggering condition, retraining of a model. The example method also includes generating, by data grafting circuitry and in response to scheduling the retraining of the model, a context-relevant training data set based on a target context vector. The example method also includes retraining, by model training circuitry, the model using the context-relevant training data set to mitigate deterioration of performance of the model.

Abstract: An example method includes detecting, by context analysis circuitry, occurrence of a triggering condition. The example method also includes scheduling, by context analysis circuitry and based on the occurrence of the triggering condition, retraining of a model. The example method also includes generating, by data grafting circuitry and in response to scheduling the retraining of the model, a context-relevant training data set based on a target context vector. The example method also includes retraining, by model training circuitry, the model using the context-relevant training data set to mitigate deterioration of performance of the model.

Abstract: Systems, apparatuses, methods, and computer program products are disclosed for mitigating deterioration of modeling system performance. An example method includes detecting, by context analysis circuitry, occurrence of a triggering condition. The example method also includes determining, by context analysis circuitry and in response to detecting the occurrence of the triggering condition, a destination modeling solution based on a regime strategy for the modeling system. The example method also includes switching, by regime deployment circuitry and based on the regime strategy, from a source modeling solution to a destination modeling solution.

Abstract: Systems, apparatuses, methods, and computer program products are disclosed for unsupervised named entity recognition. An example method includes receiving, by a communications circuitry, a reference named entity list, the reference named entity list identifying a set of named entities and an entity type of each identified named entity. The example method further includes generating, by a vectorizer, vectors from the named entities identified in the reference named entity list, and consolidating, by a synthesizer, the generated vectors into a set of representative vectors, wherein each representative vector is associated with a particular entity type. Finally, the example method receiving, by an analysis engine, a set of text, and performing, by the analysis engine, named entity recognition on the set of text using the set of representative vectors to generate a tagged set of text.

Abstract: In general, this disclosure describes techniques for generating and evaluating automatic transcripts of audio recordings containing human speech. In some examples, a computing system is configured to: generate transcripts of a plurality of audio recordings; determine an error rate for each transcript by comparing the transcript to a reference transcript of the audio recording; receive, for each transcript, a subjective ranking selected from a plurality of subjective rank categories; determine, based on the error rates and subjective rankings, objective rank categories defined by error-rate ranges; and assign an objective ranking to a new machine-generated transcript of a new audio recording, based on the objective rank categories and an error rate of the new machine-generated transcript.

Abstract: Systems, apparatuses, methods, and computer program products are disclosed for automatically determining accuracy of entity recognition of text. An example method includes segmenting a service entity recognition analysis of the text and a gold entity recognition analysis of the text into common superstrings that define entity spans. The example method further includes classifying each of the entity spans based on an accuracy of entity recognition in the service analysis of the text corresponding to the entity spans using a classification system that differentiates accurately identified but improperly bounded entities into at least three subcategories to obtain an entity accuracy classification. The example method also includes obtaining a score report based on the entity accuracy classification. The example method additionally includes performing an action set based on the entity accuracy classification.

Abstract: In general, this disclosure describes techniques for generating and evaluating automatic transcripts of audio recordings containing human speech. In some examples, a computing system is configured to: generate transcripts of a plurality of audio recordings; determine an error rate for each transcript by comparing the transcript to a reference transcript of the audio recording; receive, for each transcript, a subjective ranking selected from a plurality of subjective rank categories; determine, based on the error rates and subjective rankings, objective rank categories defined by error-rate ranges; and assign an objective ranking to a new machine-generated transcript of a new audio recording, based on the objective rank categories and an error rate of the new machine-generated transcript.

Abstract: Systems, apparatuses, methods, and computer program products are disclosed for automatically determining accuracy of entity recognition of text. An example method includes segmenting a service entity recognition analysis of the text and a gold entity recognition analysis of the text into common superstrings that define entity spans. The example method further includes classifying each of the entity spans based on an accuracy of entity recognition in the service analysis of the text corresponding to the entity spans using a classification system that differentiates accurately identified but improperly bounded entities into at least three subcategories to obtain an entity accuracy classification. The example method also includes obtaining a score report based on the entity accuracy classification. The example method additionally includes performing an action set based on the entity accuracy classification.