Abstract: A computer-implemented method, a computer program product, and a computer system for classifying relevance of training data. A computer uses labeled data from users as training data to train a relevance classifier. A computer classifies, by the relevance classifier, the labeled data from the users into a set of groups. A computer generates, by the relevance classifier, relevant training data partitioned by the set of groups. In response to receiving a query from a user, a computer selects, from the relevant training data, relevant training samples for the user, where the relevant training samples are in one or more groups to which the user belongs. A computer selects, from relevant training samples for the user, top relevant training samples for the user. A computer uses the top relevant training samples for the user to generate a prompt of a machine learning model.

Abstract: Mechanisms are provided for selecting an artificial intelligence (AI) computer model for processing an input. The mechanisms generate a distribution of characteristics of previous input data processed by the data processing system. The mechanisms receive current input data and compare characteristics of the current input data to the distribution to generate a measure of similarity. An AI computer model selection engine processes the measure of similarity to select an AI computer model from a plurality of different AI computer models. The processing of the measure of similarity includes evaluation of the measure of similarity relative to one or more threshold values. The current input data is processed by the selected AI computer model to generate a result of processing the current input data.

Abstract: Techniques regarding the governing of use by a consumer of an artificial intelligence technology are described. For example, one or more embodiments described herein can comprise a system, which can comprise a memory that can store computer executable components. The system can also comprise a processor, operably coupled to the memory, and that can execute the computer executable components stored in the memory. The computer executable components can include an analyzing component that can, based on an artificial intelligence usage policy, analyze a proposed use by a consumer of an artificial intelligence technology, resulting in an analyzed use of the artificial intelligence technology. The computer executable components can further include a governing component that can, based on the analyzing, govern use by the consumer of the artificial intelligence technology.

Abstract: An autonomous agent operating method, system, and computer program product, including launching a first autonomous agent for a task with an initial update interval and adjusting the initial update interval for a second autonomous agent based on a second task for the second autonomous agent being similar to the task over time in relation to a trust level of a human user in a performance of the first autonomous agent.

Abstract: One or more systems, devices, computer program products and/or computer-implemented methods provided herein relate to risk assessment for artificial intelligence models, and more specifically, to the generation of customized risk scores and converted comparable scores. In an embodiment, the customized risk assessment scores can be based on a risk profile determined from risk assessment requirements and measurements of an artificial intelligence model. In another embodiment, one or more customized risk assessment scores can be converted to a converted risk assessment score that is comparable to a customized risk assessment score or another converted risk assessment score.

Abstract: Provided are techniques for dynamic fact contextualization in support of AI model development. A template from a plurality of templates is selected, where the template includes definitions for identifying facts. The facts are retrieved from a facts repository based on the definitions. It is determined that that the facts are valid based on one or more policies. A FactSheet is generated using the template and the facts. A machine learning model is used to identify one or more deficient facts from the FactSheet. The FactSheet is displayed in a preview with the one or more deficient facts. One or more facts corresponding to the one or more deficient facts are located. The FactSheet is updated to correct the one or more deficient facts with the corresponding facts.

Abstract: Performance of a computer implementing a machine learning system is improved by providing, via a graphical user interface, to an annotator, unlabeled corpus data to be labeled; obtaining, via the interface, labels for the unlabeled corpus data; and detecting, with a consistency calculation routine, concurrent with the labeling, internal inconsistency and/or external inconsistency in the labeling. Responsive to the detection, intervene in the labeling with a reactive intervention subsystem until the inconsistency is addressed. The labeling is completed subsequent to the intervention; the system is trained to provide a trained machine learning system, based on results of the completed labeling; and classification of new data is carried out with the trained system. Proactive intervention schemes are also provided.

Abstract: An autonomous agent operating method, system, and computer program product, including launching a first autonomous agent for a task with an initial update interval and adjusting the initial update interval for a second autonomous agent based on a second task for the second autonomous agent being similar to the task over time in relation to a trust level of a human user in a performance of the first autonomous agent.

Abstract: A method, a computer program product, and a computer system determine when to transfer a communication session from an agent to a bot. The method includes monitoring the communication session between the agent and a user. The method includes determining a continuing utility value indicating a predicted continuing cost to maintaining the communication session with the agent. The continuing utility value is indicative of a predicted continuing benefit to maintaining the communication with the agent. The method includes determining a transferring utility value indicating a predicted transferring cost to transferring the communication session from the agent to the bot. The transferring utility value is indicative of a predicted transferring benefit to transferring the communication session from the agent to the bot. The method includes, as a result of the predicted transferring benefit being greater than the predicted continuing benefit, transferring the communication session from the agent to the bot.

Abstract: An automated analytic tool (AAT) apparatus analyzes a machine learning system (MLS). The tool comprises a processor configured to receive experiment parameters associated with an experiment performed on the MLS, and captures information from a plurality of stages of the experiment. The information comprises information regarding MLS results and choices made during the experiment. The tool automatically revise the captured information into revised information utilizing a knowledge base comprising information from prior experiments. The tool then presents the revised information to a user.

Abstract: An autonomous agent operating method, system, and computer program product, including configuring an autonomous agent for a task, launching the autonomous agent with an initial update interval, the update interval determining a frequency of the autonomous agent reporting results to a human user for review, measuring the trust level of human user in a performance of the autonomous agent, and dynamically adjusting the update interval based on this measured trust.

Abstract: A method, a computer program product, and a computer system determine when to transfer a communication session from an agent to a bot. The method includes monitoring the communication session between the agent and a user. The method includes determining a continuing utility value indicating a predicted continuing cost to maintaining the communication session with the agent. The continuing utility value is indicative of a predicted continuing benefit to maintaining the communication with the agent. The method includes determining a transferring utility value indicating a predicted transferring cost to transferring the communication session from the agent to the bot. The transferring utility value is indicative of a predicted transferring benefit to transferring the communication session from the agent to the bot. The method includes, as a result of the predicted transferring benefit being greater than the predicted continuing benefit, transferring the communication session from the agent to the bot.

Abstract: Performance of a computer implementing a machine learning system is improved by providing, via a graphical user interface, to an annotator, unlabeled corpus data to be labeled; obtaining, via the interface, labels for the unlabeled corpus data; and detecting, with a consistency calculation routine, concurrent with the labeling, internal inconsistency and/or external inconsistency in the labeling. Responsive to the detection, intervene in the labeling with a reactive intervention subsystem until the inconsistency is addressed. The labeling is completed subsequent to the intervention; the system is trained to provide a trained machine learning system, based on results of the completed labeling; and classification of new data is carried out with the trained system. Proactive intervention schemes are also provided.