Abstract: A system iteratively evaluates the target machine learning model using evaluation hyperparameter values of the target machine learning model to measure performance of the target machine learning model for different combinations of the evaluation hyperparameter values. The system trains a surrogate machine learning model using the different combinations of the evaluation hyperparameter values as features and the performance of the target machine learning model based on a corresponding combination of the evaluation hyperparameter values as labels. The system generates a feature importance vector of the surrogate machine learning model based on the training of the surrogate machine learning model, generate informed priors based on the feature importance vector, and generates the target hyperparameter values of the target machine learning model based on the informed priors.

Abstract: A computerized method for determining when an ML model in a data generation process is not stable is described. An original set of training data is applied to each ML model in a data generation process. Loss values are determined from data samples from each of the ML models. The average distance between the data samples that have a difference in loss value less than a threshold are determined for each of the ML models. The dependency rate of separation between the average distances are analyzed versus the number of model runs as an exponential model. Based on the analyzing, it is determined whether the ML models are stable or divergent.

Abstract: A computing system generates transformation error probabilities by analyzing a training data set containing training strings, each transformation error probability indicating a probability that a per-character transformation applied to a character of a training string results in a typographical error in a resulting transformation string, wherein the training data set includes strings from a historical dataset of strings including typographical errors. The computing system populates a probabilistic graphical model with the transformation error probabilities corresponding to each resulting transformation string and predicts a likelihood that an input string contains a typographical error based on the probabilistic graphical model.

Abstract: A computing device records feature embeddings of each document as a document node of a knowledge graph and connects each document node of the knowledge graph with one or more engagement edges based on engagement telemetry data indicating a measure of engagement with the documents stored in the document datastore. The computing device trains a graph neural network using the knowledge graph populated with each document node and the one or more engagement edges. The computing device may generate a feature embedding for the document query and classify one or more documents from the document datastore as relevant to the document query using the graph neural network based on the feature embedding of the document query.

Abstract: A computing system extracts test data and training data from an ordered data stream and predicts doubly robust outcomes for the ordered data stream based on a combination of treatments predicted based on controls of the ordered data stream and outcomes predicted based on the treatments and the controls of the ordered data stream. The computing system measures concept drift between the test data and the training data for the machine learning model. The concept drift is measured as an expectation of differences between the doubly robust outcomes for the ordered data stream and the doubly robust outcomes for the training data. The computing system selects retraining feature vectors from feature vectors of the ordered data stream, based on the concept drift being measured to satisfy a retraining condition and retrains the machine learning model using the retraining feature vectors.

Abstract: A computing system selects second data samples from second input data based on predictions from a surrogate membership classification model, wherein the second data samples are predicted by the surrogate membership classification model to be members of a second consistency class different from the first consistency class based on feature content of each data sample of the second input data. The computing system retrains the target machine learning model using the second data samples, based on the selecting operation.

Abstract: Systems and methods are described for identifying and resolving performance issues of automated components. The automated components are segmented into groups by applying a K-means clustering algorithm thereto based on segmentation feature values respectively associated therewith, wherein an initial set of centroids for the K-means clustering algorithm is selected by applying a set of context rules to the automated components. Then, for each group, a performance ranking is generated based at least on a set of performance feature values associated with each of the automated components in the group and a feature importance value for each of the performance features. The feature importance values are determined by training a machine learning based classification model to classify automated components into each of the groups, wherein the training is performed based on the respective performance feature values of the automated components and the respective groups to which they were assigned.

Abstract: Methods, systems, apparatuses, and computer-readable storage mediums are descried for identifying a similarity between queries. An intermediate representation generator receives a set of queries from a repository, each query in the set of queries having generated a corresponding set of data stored in a data store. An intermediate representation is generated for each query, where the intermediate representation is characterized by a feature associated with text specified in the query. A similarity determiner determines similarity scores between pairs of intermediate representations. A pair of intermediate representations with a similarity score above a threshold is identified. An indication is generated that sets of data corresponding to queries corresponding to the intermediate representations are overlapping.

Abstract: A computing system encodes a next graph based on modified source code files recorded by the next code commit event. The computing system inputs the next graph to a graph machine learning model, the graph machine learning model being trained by graphs representing modified source code files and software test results corresponding to multiple code commit events occurring prior to the next code commit event in the sequence of code commit events. The computing system determines an order of test cases of the next code commit event using the graph machine learning model in an inference mode. The computing system executes the test cases according to the order during the software development build process corresponding to the next code commit event.

Abstract: A hyperparameter tuning system generates, for each hyperparameter, a performance attribution statistic corresponding to an evaluation metric of the machine learning model based on historical experiment statistics for the evaluation metric and the machine learning model. The hyperparameter tuning system allocates a weight to each hyperparameter based on the performance attribution statistic of the hyperparameter. The hyperparameter tuning system updates, in a series of experiments, the hyperparameters based on the weight assigned to each hyperparameter and selects a set of the hyperparameters for the machine learning model from one of the experiments, wherein the set of the hyperparameters results in a recorded value of the evaluation metric that satisfies a tuning condition.

Abstract: Systems and methods are described for identifying and resolving performance issues of automated components. The automated components are segmented into groups by applying a K-means clustering algorithm thereto based on segmentation feature values respectively associated therewith, wherein an initial set of centroids for the K-means clustering algorithm is selected by applying a set of context rules to the automated components. Then, for each group, a performance ranking is generated based at least on a set of performance feature values associated with each of the automated components in the group and a feature importance value for each of the performance features. The feature importance values are determined by training a machine learning based classification model to classify automated components into each of the groups, wherein the training is performed based on the respective performance feature values of the automated components and the respective groups to which they were assigned.

Abstract: A system inputs a textual sample of the structured datastore including the unknown structure properties into a trained machine learning model, wherein the trained machine learning model is trained by textual training samples of structured training datastores with labeled structure properties corresponding to the unknown structure properties and includes a loss function corresponding to each labeled structure property. The system predicts labels for the unknown structure properties of the structured datastore using the trained machine learning model based on the textual sample. The system may parse the structured datastore based on the predicted labels or output a structured datastore formatted in compliance with the predicted structure dialect.

Abstract: Methods, systems, apparatuses, and computer-readable storage mediums are descried for identifying a similarity between queries. An intermediate representation generator receives a set of queries from a repository, each query in the set of queries having generated a corresponding set of data stored in a data store. An intermediate representation is generated for each query, where the intermediate representation is characterized by a feature associated with text specified in the query. A similarity determiner determines similarity scores between pairs of intermediate representations. A pair of intermediate representations with a similarity score above a threshold is identified. An indication is generated that sets of data corresponding to queries corresponding to the intermediate representations are overlapping.

Abstract: A computing system collects customer support information about the customer support session including a transcript of communications associated with the customer support session. The computing system generates a routing score based on the customer support information about the customer support session using a machine learning model. The machine learning model is trained by training data from other customer support sessions, the training data from each other customer support session including a root cause indication qualified by customer input characterizing an outcome of that customer support session. The computing system identifies a destination for communicating the customer support report of the customer support session based on the routing score satisfying a routing condition for the destination. The customer support report includes some of the customer support information from the customer support session.

Abstract: Described are examples for information discovery using domain-specific term matching, including obtaining, for a search session related to a domain, a set of search strings used for searching the domain for information discovery during the search session, identifying, for the search session, a string of the set of search strings as a search string hit and the remaining strings in the set of search strings as search string misses, and correlating, into a set of domain-specific search string misses, the remaining strings in the set of search strings with additional remaining search strings from one or more other search session related to the domain that have the same string identified as the search string hit.

Abstract: The described technology provides detection of a regime shift in streaming data by generating a curve fit of the streaming data representing an attribute over a duration to each of a plurality of probability density functions, scoring conformity of each curve fit to yield a plurality of fit scores, selecting a first duration probability density function among the plurality of probability density functions based on satisfaction of a probability density function fit condition by the fit score corresponding to the first duration probability density function, determining a probability density function change between the selected first duration probability density function and a second duration probability density function, wherein the second duration probability density function is selected for streaming data representing the attribute over a different duration, and indicating detection of the regime shift based on the determined probability density function change satisfying a shift condition.

Abstract: A sample of data, including a risk factor, is selected by a machine learning (ML) model of an extreme value theory (EVT) mechanism. A threshold is determined by the ML model based on the risk factor, an outlier score is generated for the sample, and the outlier score is compared to the threshold. The sample is identified as anomalous based on the generated outlier score being greater than the threshold. A schema comprising results of an investigation into the sample and the risk factor is updated based on the received schema.

Abstract: Techniques are described herein that are capable of providing privacy-preserving rules-based targeting using machine learning. Ranks are assigned to entities using a machine learning model. Values of each targetable feature associated with the respective entities are ordered. For each targetable feature, the entities are sorted among bins based on the values of the feature associated with the respective entities. For each targetable feature, a bin is selected from the bins that are associated with the feature based on the selected bin including more entities having respective ranks that are within a designated range than each of the other bins that are associated with the feature. A targeting rule is established, indicating a prerequisite for targeting an entity. The prerequisite indicating that the value of each targetable feature associated with the entity is included in a respective interval associated with the selected bin for the feature.

Abstract: Techniques are described herein that are capable of performing transfer-learning for structured data with regard to journeys defined by sets of actions. A first deep neural network (DNN) for a first journey is trained using structured data. Weights of nodes in the first DNN are transferred to nodes in a second DNN for a second journey using transfer-learning. An embedding layer replaces a final layer of the first DNN in the second DNN to provide an output with a same number of nodes as a pre-final layer of the first DNN. Weights of the nodes in the embedding layer are initialized based at least on a probability that a new feature of the second journey co-occurs with each feature in the structured data. A softmax function is applied on a final layer of the second DNN to indicate possible next actions of the second journey.

Abstract: A subset of data that includes a feature may be selected from a dataset. Parameters from the selected subset of data are determined and an extreme value theory (EVT) algorithm is implemented to determine a probability value for the feature based at least in part on the determined parameters. Based on the determined probability value for the feature, an outlier score is generated for the feature. Based on the outlier score being above a threshold, the subset is identified as anomalous.