Abstract: A process evaluates an AI assistant by collecting feedback tuples and a list of topics of the artificial intelligence assistant, wherein each feedback tuple include a question, a corresponding answer generated by the artificial intelligence assistant, and qualitative feedback corresponding to the question and the corresponding answer. A few-shot classifier outputs derived feature data. The process generates a feature-target training dataset by combining multiple derived features with metafeatures corresponding to each question-answer pair of the derived feature data and adding a quantitative feedback target corresponding to each question-answer pair of the derived feature data, and trains an attribution model using the feature-target training dataset to yield a trained attribution model. The process extracts feature importance vectors from the trained attribution model.

Abstract: One or more follow-up response recommendations relating to a question-answer event processed by an artificial intelligence assistant are generated. The one or more follow-up response recommendations identify questions from a predefined response library. A weighted average of embeddings is generated for each question-answer conversation of historical question-answer events collected via telemetry. A historical question-answer conversation includes a chronologically ordered sequence of question-answer events. A self-supervised weighted embedding dataset is generated and includes the weighted average of embeddings for each historical question-answer conversation and at least one corresponding semantic nearest neighbor question from the predefined response library. Topic weights of a follow-up question recommender are tuned by training the follow-up question recommender using the self-supervised weighted embedding dataset.

Abstract: Example aspects include techniques for anomaly detection via sparse judgmental samples. These techniques may include generating, via lexical analysis, a plurality of tokens from a textual representation of a machine learning (ML) model and generating, via a parser, based on the plurality of tokens, an abstract syntax tree (AST) corresponding to the ML model. In addition, the techniques may include identifying a data dependency of the ML model based on an AST node within the AST, the AST node corresponding to a data source and the data dependency indicating the ML model depends on the data source. Further, the techniques may include detecting a potential issue associated with the data source, and transmitting, based on the data dependency, an alert notification in response to the potential issue.

Abstract: A system iteratively evaluates the target machine learning model using evaluation hyperparameter values of the target machine learning model to measure performance of the target machine learning model for different combinations of the evaluation hyperparameter values. The system trains a surrogate machine learning model using the different combinations of the evaluation hyperparameter values as features and the performance of the target machine learning model based on a corresponding combination of the evaluation hyperparameter values as labels. The system generates a feature importance vector of the surrogate machine learning model based on the training of the surrogate machine learning model, generate informed priors based on the feature importance vector, and generates the target hyperparameter values of the target machine learning model based on the informed priors.

Abstract: A churn prediction system includes at least one hardware processor, a memory including a historical sample set of subscriber data, and a churn prediction engine executing on the at least one hardware processor. The churn prediction engine is configured to identify the historical sample set, identify a set of attributes, automatically select a subset of attributes based on an information gain value, generate a decision tree by recursively generating nodes of the decision tree by computing an information gain value for each remaining attribute of the subset of attributes, identifying a highest attribute having the highest information gain value, and assigning the highest attribute to the node. The churn prediction engine is also configured to receive target data for a target subscriber, apply the target data to the decision tree, thereby generating a churn prediction for the target subscriber, and identify the target subscriber as a churn prediction.

Abstract: Described are examples for generating a model for forecasting time series data. For a timeseries data set, one or more layers can be provided, where each layer in the one or more layers includes, for each timeseries data input in at least a portion of multiple timeseries data inputs, generating, for the timeseries data input, a short range output from a causal convolution process that is based on timeseries data inputs from the timeseries data set that are associated with timestamps within a threshold time before the timestamp of the timeseries data input, and generating, for the timeseries data input, a long range output from a transformer process based on the short range outputs from the causal convolution process for each timeseries data input from at least the portion of the multiple timeseries data inputs that are associated with timestamps before the timestamp of the timeseries data input.

Abstract: Example aspects include techniques for anomaly detection via sparse judgmental samples. These techniques may include generating, via lexical analysis, a plurality of tokens from a textual representation of a machine learning (ML) model and generating, via a parser, based on the plurality of tokens, an abstract syntax tree (AST) corresponding to the ML model. In addition, the techniques may include identifying a data dependency of the ML model based on an AST node within the AST, the AST node corresponding to a data source and the data dependency indicating the ML model depends on the data source. Further, the techniques may include detecting a potential issue associated with the data source, and transmitting, based on the data dependency, an alert notification in response to the potential issue.

Abstract: A system can implement, in a first hyperparameter configuration state, a first set of hyperparameter search operations. The first set of hyperparameter search operations includes selecting a first set of hyperparameters. Each hyperparameter of the first set of hyperparameters having a corresponding configuration. Additionally, the first set of hyperparameter search operations includes obtaining a first set of performance data that includes information indicating a performance of each hyperparameter of the first set of hyperparameters, and assigning a value to each hyperparameter of the first set of hyperparameters based on the corresponding performance data.

Abstract: This application relates to apparatus and methods for optimizing hyperparameters for machine learning models. In some examples, a computing device configures a machine learning model with a first set of hyperparameters from a pool of hyperparameters. The computing device may execute the machine learning model to generate a validation score, and may update parameters of a probability determination model based on the validation score. Further, the computing device may execute the probability determination model to generate probabilities corresponding to the first set of hyperparameters. The computing device may also determine a second set of hyperparameters from the pool of hyperparameters based on the generated probabilities, and may configure the machine learning model with the second set of hyperparameters.

Abstract: This application relates to apparatus and methods for training machine learning models. In some examples, a pool of worker pods are generated that can execute tasks to train a machine learning model. The pool of work pods are assigned tasks by a master that communicates with the worker pods using a work queue. Each worker pod can provide output using a results queue. The embodiments may operate with less reliable memory, such as object stores, which may be less costly than other types of storage mechanisms. To operate in less reliable environments, each worker pod can include a checkpoint mechanism that can recover from interruptions, such as interruptions due to node failure or preemption. For example, the checkpoint mechanism may allow a worker pod to continue processing a task, when the task is interrupted, from a last checkpoint. Processing results are provided to a results queue when a task completes.

Abstract: This application relates to apparatus and methods for optimizing hyperparameters for machine learning models. In some examples, a computing device configures a machine learning model with a first set of hyperparameters from a pool of hyperparameters. The computing device may execute the machine learning model to generate a validation score, and may update parameters of a probability determination model based on the validation score. Further, the computing device may execute the probability determination model to generate probabilities corresponding to the first set of hyperparameters. The computing device may also determine a second set of hyperparameters from the pool of hyperparameters based on the generated probabilities, and may configure the machine learning model with the second set of hyperparameters.

Abstract: A churn prediction system includes at least one hardware processor, a memory including a historical sample set of subscriber data, and a churn prediction engine executing on the at least one hardware processor. The churn prediction engine is configured to identify the historical sample set, identify a set of attributes, automatically select a subset of attributres based on an information gain value, generate a decision tree by recursively generating nodes of the decision tree by computing an information gain value for each remaining attribute of the subset of attributes, identifying a highest attribute having the highest information gain value, and assigning the highest attribute to the node. The churn prediction engine is also configured to receive target data for a target subscriber, apply the target data to the decision tree, thereby generating a churn prediction for the target subscriber, and identify the target subscriber as a churn prediction.

Abstract: A churn prediction system includes at least one hardware processor, a memory including a historical sample set of subscriber data, and a churn prediction engine executing on the at least one hardware processor. The churn prediction engine is configured to identify the historical sample set, identify a set of attributes, automatically select a subset of attributes based on an information gain value, generate a decision tree by recursively generating nodes of the decision tree by computing an information gain value for each remaining attribute of the subset of attributes, identifying a highest attribute having the highest information gain value, and assigning the highest attribute to the node. The churn prediction engine is also configured to receive target data for a target subscriber, apply the target data to the decision tree, thereby generating a churn prediction for the target subscriber, and identify the target subscriber as a churn prediction.

Abstract: A churn prediction system includes at least one hardware processor, a memory including a historical sample set of subscriber data, and a churn prediction engine executing on the at least one hardware processor. The churn prediction engine is configured to identify the historical sample set, identify a set of attributes, automatically select a subset of attributes based on an information gain value, generate a decision tree by recursively generating nodes of the decision tree by computing an information gain value for each remaining attribute of the subset of attributes, identifying a highest attribute having the highest information gain value, and assigning the highest attribute to the node. The churn prediction engine is also configured to receive target data for a target subscriber, apply the target data to the decision tree, thereby generating a churn prediction for the target subscriber, and identify the target subscriber as a churn prediction.