Abstract: An example operation may include one or more of storing a batch scoring engine and an application programming interface (API) for the batch scoring engine, receiving a trigger to perform a batch prediction process, reading input data from a source data store and executing, via the batch scoring engine, one or more predictive models on the input data to generate a predictive output and metadata associated with the predictive output, storing the predictive output and the metadata in a target data store, and updating the API with a location of the predictive output within the target data store and a location of the metadata within the target data store.

Abstract: Early indications of application programming interface (API) usage are identified by correlation to particular issues with the API including singular and mutual consistency, completeness, accuracy, and staleness. Analysis of API input and output along with data type and formatting information facilitates identification of the API issues. Establishing a correlation between API usage and issues supports early detection of potential usage reduction on a case-by-case level. Corrective action to resolve identified issues may be performed in a timely manner to maintain usage levels.

Abstract: Artificial intelligence (AI) model monitoring and ranking includes obtaining metric values indicative of performance of AI model deployments, the metric values including respective metric values measured across metrics, determining violation statuses of the metrics for each of the AI model deployments, the violation statuses indicating, for each AI model deployment, which of the metrics are violated by the AI model deployment as reflected by respective metric values for that AI model deployment, ranking the AI model deployments against each other according to a ranking model and based on the determined violation statuses for each of the AI model deployments, and providing a rank of at least some of the AI model deployments to a user.

Abstract: A processor may receive an original dataset. The processor may segment, automatically, the original dataset into a plurality of data groups. The plurality of data groups may include a model training dataset and a holdout dataset. The processor may generate a model with the model training dataset. The processor may validate the model with the holdout dataset.

Abstract: In an approach for bypassing security vulnerable and anomalous devices in a multi-device workflow, a processor monitors behavior and network traffic of a plurality of smart devices within a multi-smart device system. A processor identifies a first smart device of the plurality of smart devices with at least one of a security vulnerability and an anomaly. A processor identifies a multi-smart device workflow that includes the first smart device. A processor identifies a function of the first smart device within the multi-smart device workflow. A processor determines whether an alternative smart device can replace the first smart device within the multi-smart device workflow. Responsive to resolution of the at least one of the security vulnerability and the anomaly, a processor re-establishes the workflow with the first smart device.

Abstract: A system includes a memory having instructions therein and at least one processor in communication with the memory. The at least one processor is configured to execute the instructions to run a machine learning base model on input data to generate base model prediction data and run a machine learning error prediction model on the input data to generate error prediction data. The at least one processor is configured to execute the instructions to generate predicted correct base model prediction data based on the base model prediction data and the error prediction data. The at least one processor is configured to execute the instructions to generate confusion values data based on the base model prediction data and the predicted correct base model prediction data. The at least one processor is also configured to execute the instructions to generate base model accuracy fairness metrics data based on the confusion values data.

Abstract: Artificial intelligence (AI) model monitoring and ranking includes obtaining metric values indicative of performance of AI model deployments, the metric values including respective metric values measured across metrics, determining violation statuses of the metrics for each of the AI model deployments, the violation statuses indicating, for each AI model deployment, which of the metrics are violated by the AI model deployment as reflected by respective metric values for that AI model deployment, ranking the AI model deployments against each other according to a ranking model and based on the determined violation statuses for each of the AI model deployments, and providing a rank of at least some of the AI model deployments to a user.

Abstract: An approach is disclosed that inputs data points to a trained artificial intelligence (AI) model with an outlier model that identifies data points on which the AI model has been trained. A value is received from the outlier model corresponding to each of the data points with the received value being a prediction of whether the AI model has been trained on the respective data point. A bias analysis is performed on the trained AI model using a subset of the data points that received a prediction that indicates that the trained AI model was trained with the respective data point.

Abstract: A processor may receive, by a computing device, the indirect command from a user. The indirect command may include an instruction to the computing device to collect an information dataset from a secondary source. A processor may analyze the information dataset from the secondary source. A processor may determine one or more actions to be performed. The one or more actions may be based, at least in part, on the information dataset from the secondary source. A processor may execute the one or more actions.

Abstract: A source of bias identification (SoBI) tool is provided that identifies sources of bias in a dataset. A bias detection operation is performed on results of a computer model, based on an input dataset, to generate groupings of values for a protected attribute corresponding to a detected bias in the operation of the computer model. The SoBI tool generates a plurality of sub-groups for each grouping of values. Each sub-group comprises an individual value, or a sub-range, for the protected attribute. The SoBI tool analyzes each of the sub-groups in the plurality of sub-groups, based on at least one source of bias identification criterion, to identify one or more sources of bias in the input dataset. The SoBI tool outputs a bias notification to an authorized computing device specifying the one or more sources of bias in the input dataset.

Abstract: A machine learning model data quality improvement detection tool is provided for identifying an accurate reference group and an accurate monitored group of a machine learning model. The tool monitors a behavior of the machine learning model for a predetermined time frame. The tool compares a determined fairness metric a pre-defined fairness threshold. Responsive to the fairness metric failing to meet the pre-defined fairness threshold, the tool modifies the monitored group to include a first portion of the reference group. The tool compares a newly determined fairness metric to the pre-defined fairness threshold. Responsive to the newly determined fairness metric meeting the pre-defined fairness threshold, the tool identifies the modified monitored group including the first portion of the user-defined reference group as a new monitored group and the modified reference group without the first portion of the user-defined reference group as a new reference group.

Abstract: The method provides for one or more processor receiving on a personal device, a mixture of sounds within a sound stream from multiple sources. The one or more processors identifying one or more sounds of the mixture of sounds from the multiple sources, based on a sound separation technique. The one or more processors displaying on a user interface of the personal device an icon corresponding respectively to a classification of the one or more sounds identified from the multiple sources. The one or more processors receiving a selection of a sound from the mixture of the multiple sounds, based on an action by a user of the personal device selecting the icon displayed on the user interface of the personal device, and the one or more processors recording the sound from the mixture of the multiple sounds selected by the user.

Abstract: A processor may receive an original dataset. The processor may segment, automatically, the original dataset into a plurality of data groups. The plurality of data groups may include a model training dataset and a holdout dataset. The processor may generate a model with the model training dataset. The processor may validate the model with the holdout dataset.

Abstract: In an approach for bypassing security vulnerable and anomalous devices in a multi-device workflow, a processor monitors behavior and network traffic of a plurality of smart devices within a multi-smart device system. A processor identifies a first smart device of the plurality of smart devices with at least one of a security vulnerability and an anomaly. A processor identifies a multi-smart device workflow that includes the first smart device. A processor identifies a function of the first smart device within the multi-smart device workflow. A processor determines whether an alternative smart device can replace the first smart device within the multi-smart device workflow. Responsive to resolution of the at least one of the security vulnerability and the anomaly, a processor re-establishes the workflow with the first smart device.

Abstract: A source of bias identification (SoBI) tool is provided that identifies sources of bias in a dataset. A bias detection operation is performed on results of a computer model, based on an input dataset, to generate groupings of values for a protected attribute corresponding to a detected bias in the operation of the computer model. The SoBI tool generates a plurality of sub-groups for each grouping of values. Each sub-group comprises an individual value, or a sub-range, for the protected attribute. The SoBI tool analyzes each of the sub-groups in the plurality of sub-groups, based on at least one source of bias identification criterion, to identify one or more sources of bias in the input dataset. The SoBI tool outputs a bias notification to an authorized computing device specifying the one or more sources of bias in the input dataset.

Abstract: A system includes a memory having instructions therein and at least one processor in communication with the memory. The at least one processor is configured to execute the instructions to run a machine learning base model on input data to generate base model prediction data and run a machine learning error prediction model on the input data to generate error prediction data. The at least one processor is configured to execute the instructions to generate predicted correct base model prediction data based on the base model prediction data and the error prediction data. The at least one processor is configured to execute the instructions to generate confusion values data based on the base model prediction data and the predicted correct base model prediction data. The at least one processor is also configured to execute the instructions to generate base model accuracy fairness metrics data based on the confusion values data.

Abstract: A computer-implemented method according to one embodiment includes identifying an occurrence of accuracy drift by a trained model; identifying data associated with the accuracy drift, utilizing a drift detection model (DDM) constructed for the trained model; applying the data associated with the accuracy drift to a decision tree to determine a feature space and specific subset of the data causing the accuracy drift; analyzing a distribution of features within the feature space for the specific subset of the data causing the accuracy drift to determine specific features of the data causing the accuracy drift; and returning the specific features of the data causing the accuracy drift.

Abstract: Managing notifications is provided. Personal monitoring system inputs corresponding to each member of a defined group performing a common task are contextually analyzed to identify a notification sequence for each respective member enabling task performance in a synchronized manner. Progress of each respective member while performing activities corresponding to the common task is analyzed using the personal monitoring system inputs to enable dynamic modification of the notification sequence and content to the members in accordance with the progress. Existence of any problem is identified during performance of activities corresponding to the common task to accordingly modify the notification sequence and content to target members for mitigation of an existing problem.

Abstract: Managing notifications is provided. Personal monitoring system inputs corresponding to each member of a defined group performing a common task are contextually analyzed to identify a notification sequence for each respective member enabling task performance in a synchronized manner. Progress of each respective member while performing activities corresponding to the common task is analyzed using the personal monitoring system inputs to enable dynamic modification of the notification sequence and content to the members in accordance with the progress. Existence of any problem is identified during performance of activities corresponding to the common task to accordingly modify the notification sequence and content to target members for mitigation of an existing problem.

Abstract: For seamlessly abstracting metadata in multiple formats, an abstraction module converts first metadata of an incoming object and additional metadata for the incoming object from an annotation map contained by the first metadata into a common format useable. The additional metadata is not part of an original format of the first metadata. A communication module communicates the converted metadata to an adapter in the common format, the adapter processing the first metadata and the additional metadata independent of the original format of the first metadata.