Abstract: A method, system, and computer program product, including generating, using a linear probe, confidence scores through flattened intermediate representations and theoretically-justified weighting of samples during a training of the simple model using the confidence scores of the intermediate representations.

Abstract: Techniques for generating explanations for machine learning (ML) are disclosed. These techniques include identifying an ML model, an output from the ML model, and a plurality of constraints, and generating a plurality of neighborhoods relating to the ML model based on the plurality of constraints. The techniques further include generating a predictor for each of the plurality of neighborhoods using the ML model and the plurality of constraints, constructing a combined predictor based on combining each of the respective predictors for the plurality of neighborhoods, and creating one or more explanations relating to the ML model and the output from the ML model using the combined predictor.

Abstract: Techniques regarding determining sufficiency of one or more machine learning models are provided. For example, one or more embodiments described herein can comprise a system, which can comprise a memory that can store computer executable components. The system can also comprise a processor, operably coupled to the memory, and that can execute the computer executable components stored in memory. The computer executable components can comprise a measurement component that measures maximum deviation of a supervised learning model from a reference model over a certification set and an analysis component that determines sufficiency of the supervised learning model based at least in part on the maximum deviation.

Abstract: In an approach to improve the efficiency of solving problem instances by utilizing a machine learning model to solve a sequential optimization problem. Embodiments of the present invention receive a sequential optimization problem for solving and utilize a random initialization to solve a first instance of the sequential optimization problem. Embodiments of the present invention learning, by a computing device a machine learning model, based on a previously stored solution to the first instance of the sequential optimization problem. Additionally, embodiments of the present invention generate, by the machine learning model, one or more subsequent approximate solutions to the sequential optimization problem; and output, by a user interface on the computing device, the one or more subsequent approximate solutions to the sequential optimization problem.

Abstract: An input model can be received, along with a set of requirements. The set of requirements may describe an output model to be trained. The output model can then be trained. The training of the output model can be based on the input model and based further on at least one intermediate model.

Abstract: One or more computer processors compute a maximum likelihood path matrix comprising a respective shortest path between each state in a set of states associated with a model trained with a deep reinforcement learning policy. The one or more computer processors generate explanations for the deep reinforcement learning policy based one or more identified meta-states for each state in the set of states and corresponding selected strategic states utilizing the computed maximum likelihood path matrix.

Abstract: A method, computer program product and system are provided to generate perturbed text is provided. A processor receives a string of text from a user. A processor determines one or more classifications for at least one word in the string of text by a classification model. A processor determines a plurality of perturbations of the at least one word based on the one or more classifications, where the plurality of perturbations do not share the same one or more classifications as the least one word in the string of text. A processor selects a perturbation of the string of text based on (i) an edit distance between the string of text and the plurality of perturbations, and (ii) a fluency metric for each of the plurality of perturbations. A processor provides the perturbation of the string of text to the user.

Abstract: A method, a neural network, and a computer program product are provided that provide training of neural networks with continued fractions architectures. The method includes receiving, as input to a neural network, input data and training the input data through a plurality of continued fractions layers of the neural network to generate output data. The input data is provided to each of the continued fractions layers as well as output data from a previous layer. The method further includes outputting, from the neural network, the output data. Each continued fractions layer of the continued fractions layers is configured to calculate one or more linear functions of its respective input and to generate an output that is used as the input for a subsequent continued fractions layer, each continued fractions layer configured to generate an output that is used as the input for a subsequent layer.

Abstract: A method, computer program, and computer system are provided for providing artificial intelligence explanations. An explanation request corresponding to an output or a behavior of an artificial intelligence system is received from a user. A context or user profile associated with the user is identified. A plurality of explanation methods corresponding to the artificial intelligence system is accessed. Each explanation method provides an independent explanation for the output or the behavior of the artificial intelligence system and is rated based on a set of explanation evaluation criteria corresponding to the context or user profile. An explanation method having a highest rating is selected from among the plurality of explanation methods, and an explanation of the output or the behavior of the artificial intelligence system corresponding to the selected explanation method to the user.

Abstract: In an embodiment, a method for generating contrastive information for a classifier prediction comprises receiving image data representative of an input image, using a deep learning classifier model to predict a first classification for the input image, evaluating the input image using a plurality of classifier functions corresponding to respective high-level features to identify one or more of the high-level features absent from the input image, and identifying, from among the high-level features absent from the input image, a pertinent-negative feature that, if added to the input image, will result in the deep learning classifier model predicting a second classification for the modified input image, the second classification being different from the first classification. In an embodiment, the method includes creating a pertinent-positive image that is a modified version of the input image that has the first classification and fewer than all superpixels of the input image.

Abstract: A computer-implemented method, system, and non-transitory computer-readable storage medium for enhancing performance of a first model. The first model is trained with a training data set. A second model receives the training data set associated with the first model. The second model provides the first model with a hardness value associated with prediction of each data point of the training data set. The first model determines a confidence value regarding predicting each data point based on the training data set, and determines a ratio of the hardness value of a prediction of each data point by the second model with respect to the confidence value of the first model. The first model is retrained with a re-weighted training data set when the determined ratio is lower than a value of ?.

Abstract: Define a similarity measure between first and second points in a data space by operation of a machine learning model. Generate interpretable representations of the first and second points. Generate an interpretable local description of the similarity measure by approximating the similarity measure as a distance between the interpretable representations of the first and second points. The distance between the interpretable representations incorporates a matrix. Learn values for the matrix through optimizing a loss function evaluated on perturbations of the first and second points. Explain a value of the similarity measure between the first and second points using elements of the matrix. Assess the explanation of the value of the similarity measure using a rubric. In response to the assessment of the explanation of the value of the similarity measure, modify the machine learning model. Deploy the modified machine learning model.

Abstract: A method, system, and computer program product, including generating a contrastive explanation for a decision of a classifier trained on structured data, highlighting an important feature that justifies the decision, and determining a minimal set of new values for features that alter the decision.

Abstract: A computer-implemented method, a computer program product, and a computer system for diagnosing anomalies detected by a black-box machine learning model. A computer determines a local variance of a test sample in a test dataset, where the local variance represents uncertainty of a prediction by the black-box machine learning model. The computer initializes optimal compensations for the test sample, where the optimal compensations are optimal perturbations to test sample values of respective components of a multivariate input variable. The computer determines local gradients for the test sample. Based on the local variance and the local gradients, the computer updates the optimal compensations until convergences of the optimal compensations are reached. Using the optimal compensations, the computer diagnoses the anomalies detected by the black-box machine learning model.

Abstract: An approach to generate a path for minimally sufficient explanations for improving model understanding. Data is received from a user. The data is iteratively processed to generate minimally sufficient explanations based on the input data and the input of a subsequent explanation determination is constrained to the output of a prior explanation determination.

Abstract: A method, computer system, and a computer program product for invariant risk minimization games is provided. The present invention may include defining a plurality of environment-specific classifiers corresponding to a plurality of environments. The present invention may also include constructing an ensemble classifier associated with the plurality of environment-specific classifiers. The present invention may further include initiating a game including a plurality of players corresponding to the plurality of environments. The present invention may also include calculating a nash equilibrium of the initiated game. The present invention may further include determining an ensemble predictor based on the calculated nash equilibrium. The present invention may include deploying the determined ensemble predictor associated with the calculated nash equilibrium to make predictions in a new environment.

Abstract: In an approach to improve the efficiency of solving problem instances by utilizing a machine learning model to solve a sequential optimization problem. Embodiments of the present invention receive a sequential optimization problem for solving and utilize a random initialization to solve a first instance of the sequential optimization problem. Embodiments of the present invention learning, by a computing device a machine learning model, based on a previously stored solution to the first instance of the sequential optimization problem. Additionally, embodiments of the present invention generate, by the machine learning model, one or more subsequent approximate solutions to the sequential optimization problem; and output, by a user interface on the computing device, the one or more subsequent approximate solutions to the sequential optimization problem.

Abstract: In an embodiment, a method for generating contrastive information for a classifier prediction comprises receiving image data representative of an input image, using a deep learning classifier model to predict a first classification for the input image, evaluating the input image using a plurality of classifier functions corresponding to respective high-level features to identify one or more of the high-level features absent from the input image, and identifying, from among the high-level features absent from the input image, a pertinent-negative feature that, if added to the input image, will result in the deep learning classifier model predicting a second classification for the modified input image, the second classification being different from the first classification. In an embodiment, the method includes creating a pertinent-positive image that is a modified version of the input image that has the first classification and fewer than all superpixels of the input image.

Abstract: In an embodiment, a method for generating contrastive information for a classifier prediction comprises receiving image data representative of an input image, using a deep learning classifier model to predict a first classification for the input image, evaluating the input image using a plurality of classifier functions corresponding to respective high-level features to identify one or more of the high-level features absent from the input image, and identifying, from among the high-level features absent from the input image, a pertinent-negative feature that, if added to the input image, will result in the deep learning classifier model predicting a second classification for the modified input image, the second classification being different from the first classification. In an embodiment, the method includes creating a pertinent-positive image that is a modified version of the input image that has the first classification and fewer than all superpixels of the input image.

Abstract: A computer-implemented method, a computer program product, and a computer system for diagnosing anomalies detected by a black-box machine learning model. A computer determines a local variance of a test sample in a test dataset, where the local variance represents uncertainty of a prediction by the black-box machine learning model. The computer initializes optimal compensations for the test sample, where the optimal compensations are optimal perturbations to test sample values of respective components of a multivariate input variable. The computer determines local gradients for the test sample. Based on the local variance and the local gradients, the computer updates the optimal compensations until convergences of the optimal compensations are reached. Using the optimal compensations, the computer diagnoses the anomalies detected by the black-box machine learning model.