Abstract: A method of improving a language model and outputs of the language model. An output is received from a language model executed on an initial prompt. The output is validated by comparing the output to one or more rules. The output is determined to have failed to validate and in response, an updated prompt is generated. The updated prompt includes the output, at least one rule that caused the output to fail to validate, and an instruction to correct the output based on the at least one rule. The language model is executed on the updated prompt to generate a corrected output, which is validated by comparing the corrected output to the one or more rules. The language model is retrained on training data comprising at least the one or more rules, the output, the updated prompt, and the corrected output to yield a retrained language model.

Abstract: Aspects of the present disclosure provide techniques for training and using machine learning models to predict and present an optimal workflow to a user of a software application. An example method generally includes generating a training data set including a plurality of exemplars including features associated a user of a software application, a sequence of workflow steps presented to the user of the software application, and a reward metric. A plurality of hyperparameter sets for training a plurality of predictive models is generated. The plurality of predictive models are trained based on the plurality of hyperparameter sets. A hyperparameter set from the plurality of hyperparameter sets is selected based on performance metrics for each of the plurality of predictive models. A machine learning model is trained based on the selected hyperparameter set and the training data set, and the trained machine learning model is deployed.

Abstract: Aspects of the present disclosure provide techniques for training and using machine learning models to predict and present an optimal workflow to a user of a software application. An example method generally includes generating a plurality of sequences for a workflow, the workflow including a plurality of steps. Each respective sequence of the plurality of sequences for the workflow is deployed to a respective set of test users. A reward metric is calculated for each respective sequence based on a performance metric for users who complete the workflow and a performance metric for users who abandon the workflow or have not executed the workflow. A machine learning model is trained, based on a training data set including the plurality of sequences and the reward metric for each respective sequence, to predict an optimal workflow for a user. Generally, the machine learning model may be trained to optimize the reward metric.

Abstract: A method including applying a propensity model to a subject vector to generate a propensity value estimating a probability that a subject will perform an action. The subject vector has a data structure having features storing information regarding the subject. The method also includes applying a Shapley additive explanation tool to the propensity model to generate a subset of the features that contributed to the propensity value more than a remaining set of the features. The method also includes selecting an actionable feature from the subset of the features. The actionable feature includes a feature in the subset that an entity is able to influence. The method also includes applying a correlation model to labels for training data and the actionable feature to generate an output that describes a reason why the subject performs the action. The method also includes presenting the actionable feature and the output.

Abstract: Certain embodiments of the present disclosure provide techniques training a user detection model to identify a user of a software application based on voice recognition. The method generally includes receiving a data set including a plurality of voice interactions with users of a software application. For each respective recording in the data set, a spectrogram representation is generated based on the respective recording. A plurality of voice recognition models are trained. Each of the plurality of voice recognition models is trained based on the spectrogram representation for each of the plurality of voice recordings in the data set. The plurality of voice recognition models are deployed to an interactive voice response system.

Abstract: Aspects of the present disclosure provide techniques for training a machine learning model. Embodiments include receiving a historical support record comprising time-stamped actions, a support initiation time, and an account indication. Embodiments include determining features of the historical support record based at least on differences between times of the time-stamped actions and the support initiation time. Embodiments include determining a label for the features based on the account indication. Embodiments include training an ensemble model, using training data comprising the features and the label, to determine an indication of an account in response to input features, wherein the ensemble model comprises a plurality of tree-based models and a ranking model.

Abstract: Certain embodiments of the present disclosure provide techniques training a user detection model to identify a user of a software application based on voice recognition. The method generally includes receiving a data set including a plurality of voice interactions with users of a software application. For each respective recording in the data set, a spectrogram representation is generated based on the respective recording. A plurality of voice recognition models are trained. Each of the plurality of voice recognition models is trained based on the spectrogram representation for each of the plurality of voice recordings in the data set. The plurality of voice recognition models are deployed to an interactive voice response system.

Abstract: Certain embodiments of the present disclosure provide techniques training a user detection model to identify a user of a software application based on voice recognition. The method generally includes receiving a data set including a plurality of voice interactions with users of a software application. For each respective recording in the data set, a spectrogram representation is generated based on the respective recording. A plurality of voice recognition models are trained. Each of the plurality of voice recognition models is trained based on the spectrogram representation for each of the plurality of voice recordings in the data set. The plurality of voice recognition models are deployed to an interactive voice response system.

Abstract: Certain aspects of the present disclosure provide techniques for predicting activity within a software application using a machine learning model. An example method generally includes generating a multidimensional time-series data set from time-series data associated with activity within a software application. The multidimensional time-series data set generally includes the time-series data organized based on a plurality of time granularities. Using a machine learning model and the generated multidimensional time-series data set, activity within the software application is predicted for one or more time granularities of the plurality of time granularities. Computing resources are allocated to execute operations using the software application based on the predicted activity within the software application.

Abstract: Certain embodiments of the present disclosure provide techniques training a user detection model to identify a user of a software application based on voice recognition. The method generally includes receiving a data set including a plurality of voice interactions with users of a software application. For each respective recording in the data set, a spectrogram representation is generated based on the respective recording. A plurality of voice recognition models are trained. Each of the plurality of voice recognition models is trained based on the spectrogram representation for each of the plurality of voice recordings in the data set. The plurality of voice recognition models are deployed to an interactive voice response system.

Abstract: Aspects of the present disclosure provide techniques for expert matching though workload intelligence. Embodiments include receiving a request for a support engagement. Embodiments include receiving workload data of a plurality of experts. Embodiments include determining a workload capacity of each respective expert based on the respective workload data for the respective expert. Embodiments include determining a respective estimated completion time for the support engagement for each of the plurality of experts using a machine learning model. Embodiments include determining match scores for the support engagement and each of the plurality of experts based on the estimated completion times and the workload capacities. Embodiments include selecting a given expert of the plurality of experts to handle the support engagement based on the match scores.

Abstract: Aspects of the present disclosure provide techniques for training a machine learning model. Embodiments include receiving a historical support record comprising time-stamped actions, a support initiation time, and an account indication. Embodiments include determining features of the historical support record based at least on differences between times of the time-stamped actions and the support initiation time. Embodiments include determining a label for the features based on the account indication. Embodiments include training an ensemble model, using training data comprising the features and the label, to determine an indication of an account in response to input features, wherein the ensemble model comprises a plurality of tree-based models and a ranking model.