Abstract: A method for automatically generating documentation for an artificial intelligence model includes receiving, by a computing device, an artificial intelligence model. The computing device accesses a model facts policy that indicates data to be collected for artificial intelligence models. The computing device collects artificial intelligence model facts regarding the artificial intelligence model according to the model facts policy. The computing device accesses a factsheet template. The factsheet template provides a schema for an artificial intelligence model factsheet for the artificial intelligence model. The computing device populates the artificial intelligence model factsheet using the factsheet template with the artificial intelligence model facts related to the artificial intelligence model.

Abstract: Aspects of the present invention disclose a method, computer program product, and system for detecting and mitigating adversarial virtual interactions. The method includes one or more processors initiating a mitigation protocol on interactions between the user and the virtual agent, wherein the mitigation protocol is based on the actions performed by the user while interacting with the virtual agent. The method further includes one or more processors, in response to initiating the mitigation protocol on interactions between the user and the virtual agent, generating a lower fidelity response from the virtual agent to the user, wherein the lower fidelity response is a progressive dilution of the precision of language of an original response from the virtual agent to the user prior to the user exceeding the risk level threshold.

Abstract: A method for automatically generating documentation for an artificial intelligence model includes receiving, by a computing device, an artificial intelligence model. The computing device accesses a model facts policy that indicates data to be collected for artificial intelligence models. The computing device collects artificial intelligence model facts regarding the artificial intelligence model according to the model facts policy. The computing device accesses a factsheet template. The factsheet template provides a schema for an artificial intelligence model factsheet for the artificial intelligence model. The computing device populates the artificial intelligence model factsheet using the factsheet template with the artificial intelligence model facts related to the artificial intelligence model.

Abstract: A computing method receives a labeled sample from an annotator. The method may determine a plurality of reference model risk scores for the first labeled sample, where each reference model risk score corresponds to an amount of risk associated with adding the first labeled sample to a respective reference model of a plurality of reference models. The method may determine an overall risk score for the first labeled sample based on the plurality of reference model risk scores. The method may further determine a probe for confirmation of the first labeled sample and a trust score for the annotator by sending the probe to one or more annotators. In response to determining a trust score for the annotator the method may add the labeled sample to a ground truth or reject the labeled sample.

Abstract: Systems and methods to generate a cognitive model are described. A particular example of a system includes a memory including program code having an application programming interface and a user interface, and a processor configured to access the memory and to execute the program code to generate a cognitive model, to run analysis on the cognitive model to determine a factor that is impacting a performance of the cognitive model, to determine an action based on the factor, to report at least one of the factor and the action to a user, and to use the action to generate a second cognitive model.

Abstract: Aspects of the present invention disclose a method, computer program product, and system for detecting and mitigating adversarial virtual interactions. The method includes one or more processors initiating a mitigation protocol on interactions between the user and the virtual agent, wherein the mitigation protocol is based on the actions performed by the user while interacting with the virtual agent. The method further includes one or more processors, in response to initiating the mitigation protocol on interactions between the user and the virtual agent, generating a lower fidelity response from the virtual agent to the user, wherein the lower fidelity response is a progressive dilution of the precision of language of an original response from the virtual agent to the user prior to the user exceeding the risk level threshold.

Abstract: Aspects of the present invention disclose a method, computer program product, and system for detecting and mitigating adversarial virtual interactions. The method includes one or more processors detecting a user communication that is interacting with a virtual agent. The method further includes one or more processors determining a risk level associated with the detected user communication based on one or more actions performed by the detected user while interacting with the virtual agent. The method further includes one or more processors in response to determining that the determined risk level associated with the detected user communication exceeds a risk level threshold, initiating, a mitigation protocol on interactions between the detected user and the virtual agent, where the mitigation protocol is based on the actions performed by the detected user while interacting with the virtual agent.

Abstract: Systems and methods to generate a cognitive model are described. A particular example of a system includes a memory including program code having an application programming interface and a user interface, and a processor configured to access the memory and to execute the program code to generate a cognitive model, to run analysis on the cognitive model to determine a factor that is impacting a performance of the cognitive model, to determine an action based on the factor, to report at least one of the factor and the action to a user, and to use the action to generate a second cognitive model.

Abstract: Systems and methods to generate cognitive models are described. A particular apparatus includes a memory having program code and a processor configured to access the memory and to execute the program code to process user input that includes a natural language dialogue-based command pertaining to a cognitive model, to automatically run analysis on the cognitive model and to present a user in natural language with a recommendation to modify the cognitive model.

Abstract: Aspects of the present invention disclose a method, computer program product, and system for detecting and mitigating adversarial virtual interactions. The method includes one or more processors detecting a user communication that is interacting with a virtual agent. The method further includes one or more processors determining a risk level associated with the detected user communication based on one or more actions performed by the detected user while interacting with the virtual agent. The method further includes one or more processors in response to determining that the determined risk level associated with the detected user communication exceeds a risk level threshold, initiating, a mitigation protocol on interactions between the detected user and the virtual agent, where the mitigation protocol is based on the actions performed by the detected user while interacting with the virtual agent.

Abstract: A computing method receives a labeled sample from an annotator. The method may determine a plurality of reference model risk scores for the first labeled sample, where each reference model risk score corresponds to an amount of risk associated with adding the first labeled sample to a respective reference model of a plurality of reference models. The method may determine an overall risk score for the first labeled sample based on the plurality of reference model risk scores. The method may further determine a probe for confirmation of the first labeled sample and a trust score for the annotator by sending the probe to one or more annotators. In response to determining a trust score for the annotator the method may add the labeled sample to a ground truth or reject the labeled sample.