Abstract: A system for optimizing a machine learning model. The machine learning model generates predictions based on at least one input feature vector, each input feature vector having one or more vector values; and an optimization module with a processor and an associated memory, the optimization module being configured to: create at least one slice of the predictions based on at least one vector value, determine at least one optimization metric of the slice that is based on at least a total number of predictions for the vector value, and optimize the machine learning model based on the optimization metric.

Abstract: Techniques for determining a drift impact score in a machine learning model are disclosed. The techniques can include: obtaining a reference distribution of a machine learning model; obtaining a current distribution of the machine learning model; determining a statistical distance based on the reference distribution and the current distribution; determining a local feature importance parameter for each feature associated with a prediction made by the machine learning model; determining a cohort feature importance parameter for a cohort of multiple features based on the local feature importance parameter of each feature in the cohort; and determining a drift impact score for the cohort based on the statistical distance and the cohort feature importance parameter.

Abstract: Techniques for optimizing a machine learning model. The techniques may include obtaining multiple predictions from a machine learning model, the predictions being based on at least one input feature vector, each input feature vector having one or more vector values; creating at least one slice of the predictions based on at least one vector value; determining a sensitive bias metric for the slice based on a sensitive group; determining a base metric for the slice based on a base group; determining a parity metric for the slice based on a ratio of the sensitive bias metric and the base metric; and optimizing the machine learning model based on the parity metric.

Abstract: Techniques for optimizing a machine learning model. The techniques can include: obtaining one or more embedding vectors based on a prediction of a machine learning model; mapping the embedding vectors from a higher dimensional space to a 2D/3D space to generate one or more high density points in the 2D/3D space; clustering the high-density points by running a clustering algorithm multiple times, each time with a different set of parameters to generate one or more clusters; applying a purity metric to each cluster to generate a normalized purity score of each cluster; identifying one or more clusters with a normalized purity score lower than a threshold; and optimizing the identifying one or more clusters.

Abstract: A system for optimizing a machine learning model. The machine learning model generates predictions based on at least one input feature vector, each input feature vector having one or more vector values; and an optimization module with a processor and an associated memory, the optimization module being configured to: create at least one slice of the predictions based on at least one vector value, determine at least one optimization metric of the slice that is based on at least a total number of predictions for the vector value, and optimize the machine learning model based on the optimization metric.

Abstract: Techniques for determining a drift impact score in a machine learning model are disclosed. The techniques can include: obtaining a reference distribution of a machine learning model; obtaining a current distribution of the machine learning model; determining a statistical distance based on the reference distribution and the current distribution; determining a local feature importance parameter for each feature associated with a prediction made by the machine learning model; determining a cohort feature importance parameter for a cohort of multiple features based on the local feature importance parameter of each feature in the cohort; and determining a drift impact score for the cohort based on the statistical distance and the cohort feature importance parameter.

Abstract: Techniques for determining embedding drift score in a machine learning model. The techniques can include: obtaining one or more first embedding vectors based on at least one first prediction of a machine learning model; filtering the first embedding vectors based on a slice of the first prediction; determining a first average vector by averaging each dimension of the filtered first embedding vectors; obtaining one or more second embedding vectors on at least one second prediction of the machine learning model; filtering the second embedding vectors based on a slice of the second prediction; generating a second average vector by averaging each dimension of the filtered second embedding vectors; and determining an embedding drift score based on a distance measure of the first average vector and the second average vector.

Abstract: A system for optimizing a machine learning model. The machine learning model generates predictions based on at least one input feature vector, each input feature vector having one or more vector values; and an optimization module with a processor and an associated memory, the optimization module being configured to: create at least one slice of the predictions based on at least one vector value, determine at least one optimization metric of the slice that is based on at least a total number of predictions for the vector value, and optimize the machine learning model based on the optimization metric.

Abstract: A system for optimizing a machine learning model. The machine learning model generates predictions based on at least one input feature vector, each input feature vector having one or more vector values; and an optimization module with a processor and an associated memory, the optimization module being configured to: create at least one slice of the predictions based on at least one vector value, determine at least one optimization metric of the slice that is based on at least a total number of predictions for the vector value, and optimize the machine learning model based on the optimization metric.

Abstract: A system for optimizing a machine learning model. The machine learning model generates predictions based on at least one input feature vector, each input feature vector having one or more vector values; and an optimization module with a processor and an associated memory, the optimization module being configured to: create at least one slice of the predictions based on at least one vector value, determine at least one optimization metric of the slice that is based on at least a total number of predictions for the vector value, and optimize the machine learning model based on the optimization metric.