Abstract: Mechanisms are provided for selecting and annotating test examples for artificial intelligence (AI) model selection. First and second AI computer models process a test example of a test set to generate a corresponding first AI computer model output and second AI computer model output. A corresponding first vector embedding and second vector embedding are generated and a difference vector is generated based on a difference between the first vector embedding and the second vector embedding. The difference vectors are clustered into a plurality of clusters of difference vectors for test examples of the test set. Representative difference vectors are selected from each cluster and their corresponding test examples are annotated by an oracle for use in selecting an AI model for a task.

Abstract: Methods, systems, and computer program products for implementing active learning in NLG tasks are provided herein. A computer-implemented method includes generating multiple natural language annotations associated with multiple items of unlabeled data by processing the unlabeled data using at least one artificial intelligence model; determining at least one quality score attributed to at least a portion of the multiple generated natural language annotations based at least in part on at least one quality metric; selecting at least one of the multiple natural language annotations and at least one corresponding item of the multiple items of unlabeled data based at least in part on the at least one determined quality score; and performing one or more automated actions based at least in part on the at least one selected natural language annotation.

Abstract: A computer-implemented method including: receiving, as input, a dataset comprising training pairs (s, t), wherein each training pair comprises (i) a source sentence s and (ii) a target paraphrase t of the source sentences; at a training stage, training a machine learning model on the dataset, to obtain a trained quality-controlled paraphrase generator model, wherein during the training stage, each of the training pairs is associated with a predicted control vector representing a predicted paraphrase quality of the source sentence in the training pair; and at an inference stage, inferencing the trained quality-controlled paraphrase generator model on an input sentence, wherein the input sentence is associated with an input quality control vector, to obtain an output paraphrase of the input sentence which conforms to the quality control vector.

Abstract: An example system includes a processor to receive a list of sentiment carrying discourse markers. The processor is to select sentences in a text corpus that begin with a discourse marker from the list of sentiment carrying discourse markers followed by a comma. The processor is to remove each discourse marker and comma from a beginning of the selected sentences and labeling each of the sentences with a sentiment associated with to a corresponding removed discourse marker to generate a weakly labeled dataset. The processor is to inter-train a pretrained language model using the generated weakly labeled dataset to generate a sentiment model.

Abstract: Methods, systems, and computer program products for implementing active learning in NLG tasks are provided herein. A computer-implemented method includes generating multiple natural language annotations associated with multiple items of unlabeled data by processing the unlabeled data using at least one artificial intelligence model; determining at least one quality score attributed to at least a portion of the multiple generated natural language annotations based at least in part on at least one quality metric; selecting at least one of the multiple natural language annotations and at least one corresponding item of the multiple items of unlabeled data based at least in part on the at least one determined quality score; and performing one or more automated actions based at least in part on the at least one selected natural language annotation.

Abstract: A computer-implemented method including: receiving, as input, a dataset comprising training pairs (s, t), wherein each training pair comprises (i) a source sentence s and (ii) a target paraphrase t of the source sentences; at a training stage, training a machine learning model on the dataset, to obtain a trained quality-controlled paraphrase generator model, wherein during the training stage, each of the training pairs is associated with a predicted control vector representing a predicted paraphrase quality of the source sentence in the training pair; and at an inference stage, inferencing the trained quality-controlled paraphrase generator model on an input sentence, wherein the input sentence is associated with an input quality control vector, to obtain an output paraphrase of the input sentence which conforms to the quality control vector.

Abstract: A table-to-text (T2T) generation model provides type control and semantic diversity. A method, system, and computer program product are configured to: train a model to generate one or more logic-type-specific natural language statements based on tabular data; in response to receiving a first input comprising first input data with a user-specified logic-type, the trained model generating a first logic-type-specific natural language statement based on the first input data and the user-specified logic-type; and in response to receiving a second input comprising second input data without a user-specified logic-type, the trained model generating plural second logic-type-specific natural language statements based on the second input data, wherein respective ones of the second logic-type-specific natural language statements are generated according to respective ones of plural predefined logic-types.

Abstract: An example system includes a processor to receive a list of sentiment carrying discourse markers. The processor is to select sentences in a text corpus that begin with a discourse marker from the list of sentiment carrying discourse markers followed by a comma. The processor is to remove each discourse marker and comma from a beginning of the selected sentences and labeling each of the sentences with a sentiment associated with to a corresponding removed discourse marker to generate a weakly labeled dataset. The processor is to inter-train a pretrained language model using the generated weakly labeled dataset to generate a sentiment model.

Abstract: A computer-implemented method and apparatus for assessing treatment adherence by patients, the method comprising: receiving a model providing statistical significance of patients' response to treatment, the model based on treatment assigned to the patients, wherein the patients are diagnosed with a disease; computing by the computerized device a p-value for a result received for a patient diagnosed with the disease and being treated by the treatment, by applying the model to at least one patient; and issuing an alert responsive to the p-value being indicative of the result being unexpected beyond a threshold.