Abstract: Methods and systems disclosed herein may quantify the content and nature of first streaming data to detect when the typical composition of the first streaming data changes. Quantifying the content and nature of the first streaming data may begin by generating a baseline representation of the content of the first streaming data as represented by a first matrix. Once generated, the first matrix may be used as a control against subsequently received data streams. In this regard, a second matrix may be generated from second streaming data and compared to the first matrix to determine the differences between the first streaming data and the second streaming data. Once a difference is determined, the difference may be compared to a threshold value and, when the difference exceeds the threshold value, an administrator may be notified and corrective action taken.

Abstract: A method and related system operations include determining a predicted category by providing a prediction model with a set of input feature values and generating a plurality of conditionals based on the set of input feature values for a set of features and the predicted category. The method also includes filtering the plurality of conditionals based on a knowledge base to obtain a selected conditional by generating a set of sub-conditional paths by providing, as an input for a prompt generator model, a candidate conditional of the plurality of conditionals to the prompt generator model and selecting the candidate conditional as the selected conditional based on a determination that the set of sub-conditional paths satisfies a set of criteria associated with a set of sequences of the knowledge base. The method further includes storing the selected conditional in a data structure in association with the set of input feature values.

Abstract: Methods and systems are described herein for updating machine learning models based on the impact of features on predictions. The system inputs, into a machine learning model, a dataset including entries and features to obtain predictions. The machine learning model is trained to generate predictions for entries based on features. The system generates, for each entry, feature impact parameters indicating a relative impact of each feature on each prediction. The system determines a feature impact threshold for assessing which features have contributed to each prediction and generates, using the feature impact parameters and the feature impact threshold, a sparsity metric for each prediction. The sparsity metric indicates which features have relative impacts that meet the feature impact threshold for the prediction. The system generates a global sparsity metric for the machine learning model and updates the machine learning model based on the global sparsity metric.

Abstract: In some embodiments, a computing system may generate a prediction related to a new category (not included in a set of categories) using a machine learning model trained on a set of embeddings corresponding to the category set. As an example, the computing system may generate a set of hashes such that each hash of the hash set is mapped to an embedding of the embedding set. When a new category added to the category set, the computing system may generate a given hash for the new category and identify a first hash of the hash set that matches the given hash. Based on identifying the first hash as a matching hash, the computing system may use an existing embedding (e.g., mapped to the first hash) with the machine learning model in connection with the new category, thereby avoiding a need to add a new embedding to the embedding set.

Abstract: In some aspects, a computing system may use a machine learning model (e.g., a large language model) with a variety of query tokens as seeds to predict probabilities of future events. The system may then use the probabilities to determine the probability of a target event based on the query tokens. The system may then sort the query tokens by probability of the target event and select the top N query tokens (e.g., the top three events for a user that are predicted to be followed by the target event). The events indicated by the query tokens may be monitored for a given time period, and, if any of those events occur, the system may generate an alert.

Abstract: A method and related system of operations include providing a set of feature values to determine a first reward value to a prediction model configured with a set of model parameters and obtaining a set of feature weights for features of the set of feature values by performing a local explainability operation that comprises providing the prediction model with a set of test inputs to determine a set of feature weights. The method also includes selecting a subset of feature weights of the set of feature weights based on a feature subset of the features indicated by a policy parameter of the prediction model, determining a reward modification value based on the subset of feature weights, and determining a second reward value based on the first reward value and the reward modification value. The method also includes updating the set of model parameters based on the second reward value.

Abstract: Systems and methods for using hash tables generating textual prediction explanations. In some aspects, the system receives a first plurality of user profiles and a corresponding plurality of resource availability values. Each user profile includes values for a first set of features. The system processes a first machine learning model to extract an explainability vector. The first machine learning model receives as input the first set of features and outputs a resource availability value. The system, using the explainability vector, selects a subset of features having corresponding values in the explainability vector above a threshold. The system generates a set of categories based on the subset of features and a hash table including the set of categories. The hash table is indexable using a hash value generated based on values for the subset of features. The system transmits to a user system corresponding a user profile a textual prediction explanation.

Abstract: Systems and methods for generating lightweight or surrogate models using explainability vectors. In some aspects, the system receives a first machine learning model trained to determine resource consumption by a user system. The first machine learning takes as input a first set of features. The system processes the first machine learning model to extract an explainability vector. Based on the explainability vector, the system rearranges the first set of features to generate a second set of features. The system processes the values for the first set of features to generate values for the second set of features corresponding to user profiles and trains a second machine learning model which takes as input the second set of features.

Abstract: A method and related system operations include obtaining a time-ordered set of action types and generating a first dataset by determining, for each respective stored sequence of a plurality of stored sequences, a respective dataset element indicating that the respective stored sequence is present in the time-ordered set of action types. The method may also include generating a reduced dataset based on the first dataset by detecting that the first dataset indicates that a first sequence and a second sequence are present in the time-ordered set of action types, determining a reduced dataset element based on the detection of a presence of the first sequence and the second sequence in the time-ordered set of action types and a score between the first sequence and the second sequence indicated by a table, and detecting malicious activity using a decision model based on the reduced dataset.

Abstract: Methods and systems are described herein for generating group definition sequences for accounts (e.g., user accounts) using action sequence processing and then classifying accounts using those group definition sequences. A plurality of user account actions and corresponding time that each action was taken may be received and based on that information, a sequence of action types sometimes referred to as a time-ordered dataset of action types (e.g., based on a chronological order of the actions) may be generated. The time-ordered dataset of action types may be compared with known time-ordered sequences for a particular user group or user classification. If the time-ordered dataset of action types matches the time-ordered sequences of the particular user group, the user may be classified into that user group.

Abstract: In some aspects, a computing system may use a surrogate machine learning model to detect whether a production or other machine learning model has a tendency to generate different output depending on which subpopulation a particular sample belongs to. The surrogate machine learning model may be trained using features/outputs that are not included in the data used by the production model. For example, by using demographic information in lieu of the original labels of a dataset that was used to train a production model, a surrogate model may be used to detect whether the production model is able to discern one or more characteristics associated with but not present in a sample using other features of the dataset. Output of the surrogate machine learning model may be clustered to detect whether certain subpopulations are treated differently by the production model.

Abstract: In some aspects, a computing system may aggregating multiple counterfactual samples so that machine learning explanations can be generated for sub-populations. In addition, methods and systems described herein use machine learning and counterfactual samples to determine text to use in an explanation for a model's prediction. A computing system may also train machine learning models to not only determine whether a request to perform an action should be accepted, but also to generate output that is consistent with output generated by previous machine learning models. Further, a computing system may generate counterfactual samples based on user preferences. A computing system may obtain preferences and then apply a penalty or adjustment parameter such that when a counterfactual sample is created, the computing system is forced to change one or more features indicated by the preferences to create the counterfactual sample.

Abstract: In some aspects, a computing system may aggregating multiple counterfactual samples so that machine learning explanations can be generated for sub-populations. In addition, methods and systems described herein use machine learning and counterfactual samples to determine text to use in an explanation for a model's prediction. A computing system may also train machine learning models to not only determine whether a request to perform an action should be accepted, but also to generate output that is consistent with output generated by previous machine learning models. Further, a computing system may generate counterfactual samples based on user preferences. A computing system may obtain preferences and then apply a penalty or adjustment parameter such that when a counterfactual sample is created, the computing system is forced to change one or more features indicated by the preferences to create the counterfactual sample.

Abstract: In some aspects, a computing system may aggregating multiple counterfactual samples so that machine learning explanations can be generated for sub-populations. In addition, methods and systems described herein use machine learning and counterfactual samples to determine text to use in an explanation for a model's prediction. A computing system may also train machine learning models to not only determine whether a request to perform an action should be accepted, but also to generate output that is consistent with output generated by previous machine learning models. Further, a computing system may generate counterfactual samples based on user preferences. A computing system may obtain preferences and then apply a penalty or adjustment parameter such that when a counterfactual sample is created, the computing system is forced to change one or more features indicated by the preferences to create the counterfactual sample.

Abstract: A system including at least one processor; and at least one memory having stored thereon computer program code that, when executed by the at least one processor, controls the at least one processor to: receive an email addressed to a user; separate the email into a plurality of email components; analyze, using respective machine-learning techniques, each of the plurality of email components; feed the analysis of each of the plurality of email components into a stacked ensemble analyzer; and based on an output of the stacked ensemble analyzer, determine whether the email is malicious.

Abstract: A computing system may generate a first set of importance metrics (e.g., scores or values) for a model. The importance metrics may be generated using an explainable artificial intelligence technique, and an individual importance metric may indicate how influential a corresponding feature is for a decision made by a model. The computing system may determine an important feature and create a modified dataset by removing the important feature from the dataset. The computing system may train the model on the modified dataset and evaluate the performance of the model to determine the effect of removing the feature (e.g., which may indicate how important the feature is to output generated by the model). This process may be repeated for additional features and additional performance metrics may be obtained.

Abstract: Methods and systems disclosed herein may quantify a representation of a type of input an image analysis system should expect. The image analysis system may be trained on the type of input the image analysis system should expect using a first image stream. A first model of the type of input that the image analysis system should expect may be built from the first image stream. After the first model is built, a second image, or a second image stream, may be compared to the first model to determine a difference between the second image, or second image stream, and the first image stream. When the difference is greater than or equal to a threshold, a drift may be detected and steps may be taken to determine the cause of the drift.

Abstract: Systems and methods are disclosed herein for improving machine learning of a data set. In one example, the method may include training a predictive model on an initial data set comprising labeled data, wherein the training is performed in an active learning system. The method may further include generating a set of parameters based on the training and introducing an unlabeled data set into the predictive model. According to some embodiments, the method may further include applying the set of parameters to the unlabeled data set, generating a set of predictions associated with the applied set of parameters and calculating a first uncertainty score and a second uncertainty score associated with the generated set of predictions. Moreover, the method may also include modifying the data set based on the first uncertainty score, and modifying the predictive model based on the second uncertainty score.

Abstract: Methods and systems disclosed herein may quantify a representation of a type of input an image analysis system should expect. The image analysis system may be trained on the type of input the image analysis system should expect using a first image stream. A first model of the type of input that the image analysis system should expect may be built from the first image stream. After the first model is built, a second image, or a second image stream, may be compared to the first model to determine a difference between the second image, or second image stream, and the first image stream. When the difference is greater than or equal to a threshold, a drift may be detected and steps may be taken to determine the cause of the drift.

Abstract: Methods and systems disclosed herein may quantify a representation of a type of input an image analysis system should expect. The image analysis system may be trained on the type of input the image analysis system should expect using a first image stream. A first model of the type of input that the image analysis system should expect may be built from the first image stream. After the first model is built, a second image, or a second image stream, may be compared to the first model to determine a difference between the second image, or second image stream, and the first image stream. When the difference is greater than or equal to a threshold, a drift may be detected and steps may be taken to determine the cause of the drift.