Abstract: A method of providing a surrogate program for a program endpoint includes: obtaining, by a processor set, a set of plural input/output pairs generated using the program endpoint; generating, by the processor set, transformations based on the input/output pairs; generating, by the processor set, a model that classifies inputs of the input/output pairs to ones of the transformations based on parameters of one or more strings of the inputs; receiving, by the processor set, a new input; selecting, by the processor set and using the model, one of the transformations based on parameters of one or more strings of the new input; and generating, by the processor set, a new output by applying the selected one of the transformations to the new input.

Abstract: An embodiment for identifying and sorting duplicate datasets within a large pool of heterogeneous datasets may include received a plurality of heterogeneous datasets. The embodiment may automatically compare schema information and metadata within each of the received plurality of heterogeneous datasets to generate name-based similarity scores for each dataset. The embodiment may also automatically compare data distribution information within each of the received plurality of heterogeneous datasets to generate a plurality of data distribution similarity scores for each heterogeneous dataset. The embodiment may further include automatically calculating an overall distance metric using the name-based similarity scores and plurality of data distribution similarity scores. The embodiment may also include based on the calculate overall distance metric, automatically generating distance graphs that identifying clusters of similar datasets and illustrate inferred lineage for the clusters of similar datasets.

Abstract: A system, computer program product, and method are presented for providing replacement data for data in a time series data stream that has issues indicative of errors, where the data issues and the replacement data are related to one or more KPIs. The method includes determining one or more predicted replacement values for potentially erroneous data instances in the time series data stream. The method further includes resolving the potentially erroneous data instances with one predicted replacement value of the one or more predicted replacement values in the time series data stream.

Abstract: Provided are techniques for global context explainers for Artificial Intelligence systems using multivariate timeseries data. Predictions for multivariate timeseries data are received. Feature importance weights are generated from the predictions using a feature-based local explainer, where each of the feature importance weights is associated with a time period and a corresponding data source of timeseries data of the multivariate timeseries data. A dataset is generated using the feature importance weights, where the dataset includes, for each time period and the corresponding data source, a label indicating whether the feature importance weight is one of positive and negative. One or more global explanations are generated using the dataset and a directly interpretable rule-based explainer, where the one or more global explanations indicate how the predictions change at particular times in the multivariate timeseries data based on values from the corresponding data source.

Abstract: An embodiment for identifying and sorting duplicate datasets within a large pool of heterogeneous datasets may include received a plurality of heterogeneous datasets. The embodiment may automatically compare schema information and metadata within each of the received plurality of heterogeneous datasets to generate name-based similarity scores for each dataset. The embodiment may also automatically compare data distribution information within each of the received plurality of heterogeneous datasets to generate a plurality of data distribution similarity scores for each heterogeneous dataset. The embodiment may further include automatically calculating an overall distance metric using the name-based similarity scores and plurality of data distribution similarity scores. The embodiment may also include based on the calculate overall distance metric, automatically generating distance graphs that identifying clusters of similar datasets and illustrate inferred lineage for the clusters of similar datasets.

Abstract: Methods, systems, and computer program products for path-coverage directed black box application programming interface (API) testing are provided herein. A computer-implemented method includes determining constraints based on inputs and corresponding outputs of an API in a production environment; generating initial test inputs based at least in part on the constraints; creating a program dependency graph based on trace sequences and request-response data obtained in response to providing the initial test inputs to an endpoint of the API; enhancing the program dependency graph by generating additional test inputs directed to one or more paths of the dependency graph; identifying, based on the enhanced program dependency graph, at least a portion of the API that is not covered by an existing test suite; and using the enhanced program dependency graph to generate new test cases for the test suite based on the identifying.

Abstract: Methods, systems, and computer program products for automatically modifying responses from generative models using artificial intelligence techniques are provided herein. A computer-implemented method includes obtaining data pertaining to at least one conversation involving at least one automated conversation exchange software program and at least one user; identifying, among words proposed by the at least one automated conversation exchange software program in connection with the at least one conversation, words qualifying as belonging to one or more predetermined categories by processing the obtained data using artificial intelligence techniques; determining, by processing the identified words and at least one word-based data source, one or more alternate words; modifying at least a portion of the proposed words by replacing at least a portion of the identified words with at least a portion of the one or more alternate words; and performing at least one automated action based on the modifying.

Abstract: A post-processing method, system, and computer program product for post-hoc improvement of instance-level and group-level prediction metrics, including training a bias detector that learns to detect a sample that has an individual bias greater than a predetermined individual bias threshold value with constraints on a group bias, applying the bias detector on a run-time sample to select a biased sample in the run-time sample having a bias greater than the predetermined individual bias threshold bias value, and suggesting a de-biased prediction for the biased sample.

Abstract: According to one embodiment of the present invention, a system for modifying content associated with an item comprises at least one processor. Features of interest of the item to a plurality of different groups are determined based on user comments produced by members of the plurality of different groups. The members within each group have a common characteristic. The features of interest to each group within the content associated with the item are identified, and the content associated with the item is modified by balancing the features of interest to the plurality of different groups within the content associated with the item. Embodiments of the present invention further include a method and computer program product for modifying content associated with an item in substantially the same manner described above.

Abstract: A post-processing method, system, and computer program product for post-hoc improvement of instance-level and group-level prediction metrics, including training a bias detector on a payload data that learns to detect a sample in a customer model that has an individual bias greater than a predetermined individual bias threshold value with constraints on a group bias, suggesting, in the run-time, a de-biased prediction based on the selected biased sample by a de-biasing procedure, and an arbiter decides based on user feedback whether to use the de-biased prediction or an original prediction made prior to the de-biasing procedure from the customer model which is then used as an output.

Abstract: Methods, systems, and computer program products for automatically testing AI models in connection with enterprise-related properties are provided herein.

Abstract: Methods, systems and computer program products for automated testing of dialog systems are provided herein. A computer-implemented method includes receiving information pertaining to a given conversation workspace of an automated dialog system and identifying test case inputs to the automated dialog system, the test case inputs comprising user input for the given conversation workspace that has portions thereof modified and which the automated dialog system maps to a different intent and/or a different entity relative to the user input. The method further includes generating human-interpretable explanations of mappings of portions of the test case inputs to the different intent and/or entity, generating suggestions for modifying intents, entities and dialog flows of the given conversation workspace such that the test case inputs map to the same intent and/or the same entity as their corresponding user input, and outputting the suggestions and the human-interpretable explanations to a user.

Abstract: One embodiment provides a method, including: providing, from the central server to a machine-learning model, a training set of samples having known values for at least one target protected attribute, wherein the training set includes a first set of samples having a first value for the at least one target protected attribute and a second set of samples having a second value for the at least one target protected attribute; receiving, at the central server from the machine-learning model, an output classification for each of the samples within the training set of samples; and generating, at the central server using the output classification, a set of rules delineating a region within the machine-learning model as discriminatory, wherein the region includes a classification region where the machine-learning model classifies received samples differently based upon a value of the at least one protected attribute.

Abstract: Methods, systems, and computer program products for active explanation guided learning are provided herein. A computer-implemented method includes identifying a subset of training examples, from a set of training examples, based on at least one of (i) an uncertainty metric computed for each one of the training examples and (ii) an influence metric computed for each one of the training examples; outputting said subset of training examples to a user; obtaining, from the user, a user explanation for each training example in said subset of training examples, wherein each of the user explanations identifies at least one part of the corresponding training example; and training a machine learning model based at least in part on the user explanations, wherein said training comprises prioritizing the identified parts of the training examples in the subset.

Abstract: Methods, systems, and computer program products for determining data representative of bias within a model are provided herein. A computer-implemented method includes obtaining a first dataset on which a model was trained, wherein the first dataset contains protected attributes, and a second dataset on which the model was trained, wherein the protected attributes have been removed from the second dataset; identifying, for each of the one or more protected attributes in the first dataset, one or more attributes in the second dataset correlated therewith; determining bias among at least a portion of the identified correlated attributes; and outputting, to at least one user, identifying information pertaining to the one or more instances of bias.

Abstract: Methods, systems, and computer program products for path-coverage directed black box application programming interface (API) testing are provided herein. A computer-implemented method includes determining constraints based on inputs and corresponding outputs of an API in a production environment; generating initial test inputs based at least in part on the constraints; creating a program dependency graph based on trace sequences and request-response data obtained in response to providing the initial test inputs to an endpoint of the API; enhancing the program dependency graph by generating additional test inputs directed to one or more paths of the dependency graph; identifying, based on the enhanced program dependency graph, at least a portion of the API that is not covered by an existing test suite; and using the enhanced program dependency graph to generate new test cases for the test suite based on the identifying.

Abstract: A computer-implemented method comprising, automatically: analyzing a machine learning dataset which comprises multiple datapoints, to deduce constraints on features of the datapoints; generating a first set of CSP (Constraint Satisfaction Problem) rules expressing the constraints; based on a machine learning model which was trained on the dataset, generating a second set of CSP rules that define one or more perturbation candidates among the features of one of the datapoints; formulating a CSP based on the first and second sets of CSP rules; solving the formulated CSP using a solver; and using the solution of the CSP as a counterfactual explanation of a prediction made by the machine learning model with respect to the one datapoint.

Abstract: One embodiment provides a method, including: receiving a dataset and a model corresponding to a bias checker, wherein the bias checker detects bias within both the dataset and the model, based upon a bias checking algorithm and a bias checking policy, wherein the dataset comprises a plurality of attributes; testing the bias checking algorithm of the bias checker by (i) generating test cases that modify the dataset by introducing bias therein and (ii) running the bias checker against the modified dataset; testing the bias checking policy of the bias checker by generating a plurality of test cases and running the bias checker against the plurality of test cases; and providing a notification to a user regarding whether the bias checker failed to indicate bias for one or more of the plurality of attributes.

Abstract: Methods, systems, and computer program products for generating a data migration plan for in-place encryption of data are provided herein. A computer-implemented method includes receiving, from a user, a request to generate a migration plan for performing in-place encryption of data within a database, wherein the migration plan indicates periods of time in which portions of the data are to be encrypted; determining a set of constraints for performing the in-place encryption; generating the migration plan based at least in part on the set of constraints; and performing the in-place encryption of the data in accordance with the migration plan such that only a single copy of each of the portions is maintained during the in-place encryption, wherein the single copy comprises one of a plaintext copy of data corresponding to the portion, and an encrypted copy of data corresponding to the portion.

Abstract: Methods, systems, and computer program products for generating explanations for a semantic parser are provided herein. A computer-implemented method includes providing to a generative model (i) at least one query and (ii) a context of at least one dataset applicable to the at least one query, wherein the generative model generates a plurality of perturbations for the at least one input query based on the context; providing the plurality of perturbations as inputs to a context aware sequence-to-sequence model, thereby obtaining a plurality of outputs; and generating, for (i) an additional query provided as input to the context aware sequence-to-sequence model and (ii) a context applicable to the additional query, an explanation indicative of one or more parts of the additional query that contributes to an output corresponding to the additional query, based at least in part on the plurality of outputs corresponding to the perturbations.