Abstract: Systems and methods are disclosed for managing a generative artificial intelligence (AI) model to improve performance. Managing the generative AI model includes using a second generative AI model to generate outputs from similar inputs and comparing the outputs of the generative AI models to determine their similarity. The second generative AI model is trained using fresher training data but may not be manually supervised and adjusted to the level of the generative AI model being managed. As such, an output of the second generative AI model is compared to an output of the managed generative AI model by a classification model to determine a relevance of the output from the managed generative AI model. An output of the classification model is used to perform various suitable policies to optimize the performance of the managed generative AI model, such as providing alternate outputs, preventing providing the output, or retraining the model.

Abstract: Embodiments disclosed herein may extract trending topics from phone call transcripts or any type of text data. The phone call transcripts may be collected for a time period and the time period may be divided into time spans. For each time span having more than a threshold number of phone call transcripts, n-grams from the phone call transcripts may be extracted. The extracted n-grams may be contextually clustered by converting the n-grams into their embedding vectors, reducing the dimensionality of the embedding vectors, and clustering similar reduced dimensionality embedding vectors. Normalized occurrences of one or more clusters may be generated. The recent mean of the number of occurrences of the normalized clusters may be compared with the historical mean and offset by historical standard deviation to generate a modified Z-score. N-grams corresponding to the clusters with high Z-scores may be identified as trending topics.

Abstract: A method of score prediction uses hierarchical attention. Word features, positioning features, participant embedding features, and metadata are extracted from a transcript of a conversation. A word encoder vector is formed by multiplying weights of a word encoder layer to one or more word features. A sentence vector is formed by multiplying weights of a word attention layer to word encoder vectors. An utterance encoder vector is formed by multiplying weights of an utterance encoder layer to the sentence vector. A conversation vector is formed by multiplying weights of an utterance attention layer to utterance encoder vectors. The utterance encoder vector is combined with one or more positioning features and one or more participant embedding features. A predicted net promoter score is generated by multiplying weights of an output layer to the conversation vector combined with the metadata. The predicted net promoter score is presented in a list of conversations.