Abstract: In various embodiments, a process includes receiving input records including tabular data, where the input records are unlabeled for a concept-explainability task. The process includes obtaining primitives for at least a subset of the input records, where the obtained primitives are based at least on at least one annotation including a plurality of user-defined concept labels. The process includes training, using hardware processor(s), a plurality of candidate models using the obtained primitives. For each of the plurality of user-defined concept labels, at least one corresponding model from the plurality of candidate models is used to determine a corresponding concept labeling model. The process includes using the determined corresponding concept labeling models to label the input records with which to train a concept-explainability machine learning model using the labeled input records.

Abstract: In various embodiments, a process for providing a surrogate hierarchical multi-task machine learning model (“model”) includes configuring the model to perform (i) a knowledge distillation task associated with a pre-trained classifier (“black-box model”) and (ii) an explanation task to predict semantic concepts for explainability associated with the distillation task. The model includes a concept layer to perform the explanation task and a decision layer to perform the distillation task. The output of the concept layer is utilized as an input to the decision layer. The process includes receiving training data including input records and concept labels, and training the model by minimizing a joint loss function that combines a loss function associated with the distillation task and one associated with the explanation task. The loss function associated with the distillation task is determined by comparing an output of the decision layer and an output of the black-box model.

Abstract: A labeling function associated with generating one or more semantic concepts is received. The received labeling function is used to automatically annotate an existing dataset with the one or more semantic concepts to generate an annotated noisy dataset. A reference dataset annotated with the one or more semantic concepts is received. A training dataset is prepared including by combining at least a portion of the reference dataset with at least a portion of the annotated noisy dataset. The training dataset is used to train a multi-task machine learning model configured to perform both a decision task to predict a decision result and an explanation task to predict a plurality of semantic concepts for explainability associated with the decision task.

Abstract: A multi-task hierarchical machine learning model is configured to perform both a decision task to predict a decision result and an explanation task to predict a plurality of semantic concepts for explainability associated with the decision task, wherein a semantic layer of the machine learning model associated with the explanation task is utilized as an input to a subsequent decision layer of the machine learning model associated with the decision task. Training data is received. The multi-task hierarchical machine learning model is trained using the received training data.

Abstract: A labeling function associated with generating one or more semantic concepts is received. The received labeling function is used to automatically annotate an existing dataset with the one or more semantic concepts to generate an annotated noisy dataset. A reference dataset annotated with the one or more semantic concepts is received. A training dataset is prepared including by combining at least a portion of the reference dataset with at least a portion of the annotated noisy dataset. The training dataset is used to train a multi-task machine learning model configured to perform both a decision task to predict a decision result and an explanation task to predict a plurality of semantic concepts for explainability associated with the decision task.

Abstract: A multi-task hierarchical machine learning model is configured to perform both a decision task to predict a decision result and an explanation task to predict a plurality of semantic concepts for explainability associated with the decision task, wherein a semantic layer of the machine learning model associated with the explanation task is utilized as an input to a subsequent decision layer of the machine learning model associated with the decision task. Training data is received. The multi-task hierarchical machine learning model is trained using the received training data.

Abstract: A data stream is received. Data elements of the data stream are analyzed using one or more machine learning models and one or more machine learning prediction explanation implementations. Different candidate presentations are tested. The different candidate presentations are associated with machine learning results provided to different reviewers in a group of human-in-the-loop reviewers that review predictions of the one or more machine learning models. The different candidate presentations include different explanations generated by the one or more machine learning prediction explanation implementations and at least one control candidate presentation corresponding to an absent explanation. Different aspects of the testing are monitored. Results of the monitoring are used to make a selection among the different candidate presentations.