Abstract: One or more systems, devices, computer program products and/or computer-implemented methods of use provided herein relate to a process to facilitate abnormal document self-discovery. A system can comprise a memory that stores computer executable components, and a processor that executes the computer executable components stored in the memory, wherein the computer executable components can comprise a fairness component, an identification component, a removal component, and an evaluation component. The fairness component can receive a dataset for a tree-based model and calculates a first fairness, and the identification component can identify root to leaf paths in the tree-based model for one or more records of the dataset and one or more corresponding perturbed records. The removal component can remove at least one record of the one or more records having similar root to leaf paths to the other records of the one or more records.

Abstract: An example operation may include one or more of storing a batch scoring engine and an application programming interface (API) for the batch scoring engine, receiving a trigger to perform a batch prediction process, reading input data from a source data store and executing, via the batch scoring engine, one or more predictive models on the input data to generate a predictive output and metadata associated with the predictive output, storing the predictive output and the metadata in a target data store, and updating the API with a location of the predictive output within the target data store and a location of the metadata within the target data store.

Abstract: Early indications of application programming interface (API) usage are identified by correlation to particular issues with the API including singular and mutual consistency, completeness, accuracy, and staleness. Analysis of API input and output along with data type and formatting information facilitates identification of the API issues. Establishing a correlation between API usage and issues supports early detection of potential usage reduction on a case-by-case level. Corrective action to resolve identified issues may be performed in a timely manner to maintain usage levels.

Abstract: One example of a system comprises using a processor for identifying a model to be validated that is stored in a repository; automatically computing and recording one or more model metrics for the model to be validated in a tamper-proof manner; comparing the computed tamper-proof metrics with one or more encoded rules and policies to determine if the model to be validated complies with the one or more encoded rules and policies; and outputting a notification to a device indicating a validation status of the model to be validated based on the comparison of the computed tamper-proof metrics with the one or more encoded rules and policies.

Abstract: Artificial intelligence (AI) model monitoring and ranking includes obtaining metric values indicative of performance of AI model deployments, the metric values including respective metric values measured across metrics, determining violation statuses of the metrics for each of the AI model deployments, the violation statuses indicating, for each AI model deployment, which of the metrics are violated by the AI model deployment as reflected by respective metric values for that AI model deployment, ranking the AI model deployments against each other according to a ranking model and based on the determined violation statuses for each of the AI model deployments, and providing a rank of at least some of the AI model deployments to a user.

Abstract: A processor may receive an original dataset. The processor may segment, automatically, the original dataset into a plurality of data groups. The plurality of data groups may include a model training dataset and a holdout dataset. The processor may generate a model with the model training dataset. The processor may validate the model with the holdout dataset.

Abstract: One example of a system comprises using a processor for identifying a model to be validated that is stored in a repository; automatically computing and recording one or more model metrics for the model to be validated in a tamper-proof manner; comparing the computed tamper-proof metrics with one or more encoded rules and policies to determine if the model to be validated complies with the one or more encoded rules and policies; and outputting a notification to a device indicating a validation status of the model to be validated based on the comparison of the computed tamper-proof metrics with the one or more encoded rules and policies.

Abstract: A system includes a memory having instructions therein and at least one processor in communication with the memory. The at least one processor is configured to execute the instructions to run a machine learning base model on input data to generate base model prediction data and run a machine learning error prediction model on the input data to generate error prediction data. The at least one processor is configured to execute the instructions to generate predicted correct base model prediction data based on the base model prediction data and the error prediction data. The at least one processor is configured to execute the instructions to generate confusion values data based on the base model prediction data and the predicted correct base model prediction data. The at least one processor is also configured to execute the instructions to generate base model accuracy fairness metrics data based on the confusion values data.

Abstract: One example of a method comprises identifying a model to be validated that is stored in a repository; automatically computing and recording one or more model metrics for the model to be validated in a tamper-proof manner; comparing the computed tamper-proof metrics with one or more encoded rules and policies to determine if the model to be validated complies with the one or more encoded rules and policies; and outputting a notification to a device indicating a validation status of the model to be validated based on the comparison of the computed tamper-proof metrics with the one or more encoded rules and policies.

Abstract: Artificial intelligence (AI) model monitoring and ranking includes obtaining metric values indicative of performance of AI model deployments, the metric values including respective metric values measured across metrics, determining violation statuses of the metrics for each of the AI model deployments, the violation statuses indicating, for each AI model deployment, which of the metrics are violated by the AI model deployment as reflected by respective metric values for that AI model deployment, ranking the AI model deployments against each other according to a ranking model and based on the determined violation statuses for each of the AI model deployments, and providing a rank of at least some of the AI model deployments to a user.

Abstract: Disclosed embodiments provide techniques for monitoring application program interface (API) health based on design-time checks and runtime performance. The health of an API can include multiple factors. The factors may include correctness of the data returned/output by the API. A high amount of incorrect data may correlate to a lower health metric. A low amount of incorrect data may correlate to a higher health metric. If a health metric value falls below a predetermined threshold, it may indicate that the API health is poor, and hence, should not be used, or should be used with caution, depending on the application. In embodiments, an alternate API may be automatically selected as a replacement for an API in response to a health metric falling below a predetermined threshold.

Abstract: An artificial intelligence model that performs operating the artificial intelligence model, which data taken collectively is uncollected payload data, storing the uncollected payload data to obtain a collected payload data set in the form of a plurality of data points, clustering the plurality of data points of payload data, calculating an average feature distance, calculating average label distance, grouping all given pairs of data points, and determining a plurality of close pairs of data points.

Abstract: An approach is disclosed that inputs data points to a trained artificial intelligence (AI) model with an outlier model that identifies data points on which the AI model has been trained. A value is received from the outlier model corresponding to each of the data points with the received value being a prediction of whether the AI model has been trained on the respective data point. A bias analysis is performed on the trained AI model using a subset of the data points that received a prediction that indicates that the trained AI model was trained with the respective data point.

Abstract: A computer-implemented method includes: receiving, by a computing device and from a user device, a request to validate an application program interface (API); validating, by the computing device, the API by performing a fetch analysis using different user profiles; returning, by the computing device and to the user device, a result of the fetch analysis; validating, by the computing device, the API by performing an insert/update analysis using the different user profiles; and returning, by the computing device and to the user device, a result of the insert/update analysis

Abstract: An approach is provided in which the approach receives scored records that include a selected scored record comprising a first fairness group attribute, a first prediction, and a first confidence value corresponding to the first prediction. The approach perturbs the selected scored record to a second fairness group attribute in response to determining that the first confidence value is below a confidence threshold. The approach scores the perturbed record to generate a second prediction and a second confidence value corresponding to the second prediction. The approach modifies the selected scored record by changing the first prediction to the second prediction in response to determining that the second prediction is different from the first prediction and that the second confidence value is higher than the confidence threshold.

Abstract: A system and related method score a fairness of an outcome model. The method comprises receiving a set of original transaction records (OTRs), and selecting an OTR subset of the OTRs according to a subset selection criteria in order to reduce a number of OTRs to send to outcome model. For each OTR in the subset a perturbed transaction record (PTR) is created based on the OTR that includes changing at least one attribute in the PTR from the OTR, sending the OTR and the PTR to the outcome model, receiving an OTR outcome and a PTR outcome from the outcome model, and determining a record bias score for the OTR outcome and the PTR outcome respectively that indicates bias in the respective outcome. The OTR and the PTR bias score are stored in a bias determination system (BDS) database.

Abstract: A machine learning model data quality improvement detection tool is provided for identifying an accurate reference group and an accurate monitored group of a machine learning model. The tool monitors a behavior of the machine learning model for a predetermined time frame. The tool compares a determined fairness metric a pre-defined fairness threshold. Responsive to the fairness metric failing to meet the pre-defined fairness threshold, the tool modifies the monitored group to include a first portion of the reference group. The tool compares a newly determined fairness metric to the pre-defined fairness threshold. Responsive to the newly determined fairness metric meeting the pre-defined fairness threshold, the tool identifies the modified monitored group including the first portion of the user-defined reference group as a new monitored group and the modified reference group without the first portion of the user-defined reference group as a new reference group.

Abstract: An approach is provided for optimizing data fetching. A query employing a method to fetch data from a JSON document is received. An amount of time required to execute the query and a number of nested layers in a traversal of the JSON document required to fetch the data are determined. Based on the amount of time and the number of nested layers, a cost associated with an execution of the query is calculated. The cost is determined to exceed a threshold value. Responsive to the determination that the cost exceeds the threshold value and using historical query patterns and historical query execution times, a schema of the JSON document is re-designed. The data is fetched from the JSON document using the re-designed schema.

Abstract: A computer device receives historical prediction data, where the historical prediction data includes historical data and corresponding predictions generated for the historical data by a regression machine learning model. The computing device identifies undesired predictions in the historical prediction data based, at least in part, on a perturbation analysis, where the perturbation analysis includes modifying an attribute of the historical data and using the regression machine learning model to generate predictions for the historical data with the modified attribute. The computing device trains a binary classification model to classify predictions as undesired, using the historical prediction data and the identified undesired predictions as training data. The computing device generates a prediction for a new data entry utilizing the regression machine learning model and the binary classification model.

Abstract: One example of a method comprises identifying a model to be validated that is stored in a repository; automatically computing and recording one or more model metrics for the model to be validated in a tamper-proof manner; comparing the computed tamper-proof metrics with one or more encoded rules and policies to determine if the model to be validated complies with the one or more encoded rules and policies; and outputting a notification to a device indicating a validation status of the model to be validated based on the comparison of the computed tamper-proof metrics with the one or more encoded rules and policies.