Abstract: A method may include obtaining a trained machine learning (ML) pipeline skeleton model configured to predict functional blocks within a new ML pipeline based on meta-features of a dataset associated with the new ML pipeline; obtaining parametric templates, each of the parametric templates including fillable portions and static text portions that in combination describe a given functional block; receiving a request to generate the new ML pipeline; determining functional blocks to populate the new ML pipeline based on the pipeline skeleton model; extracting decision-making conditions leading to the functional blocks; generating explanations of the functional blocks using the parametric templates, where at least one of the fillable portions is filled based on the decision-making conditions leading to the functional blocks; instantiating the new ML pipeline including the functional blocks with the generated explanations.

Abstract: According to one or more embodiments, operations may include storing existing machine learning (ML) projects in a corpus. The operations may also include generating a search query for a new ML project based on a new dataset and a new ML task for the new ML project. In addition, the operations may include searching through the existing ML projects stored in the corpus, based on the search query, for a set of existing ML projects. Moreover, the operations may include merging the ML pipelines of the set of existing ML projects to generate a new ML pipeline for the new ML project. In addition, the operations may include adapting functional blocks of the new ML pipeline for the new ML project to enable the new ML pipeline to be executed to perform the new ML task on the new dataset.

Abstract: According to an aspect of an embodiment, operations may include predicting, by a pre-trained DNN, a first class for a first datapoint of a first dataset. A first set of feature scores is determined for the first datapoint based on the first class associated with the first datapoint. A set of confusing class pairs associated with the DNN is identified based on the first class and a predetermined class of the first datapoint. The first dataset is clustered into one of a set of semantic classes based on the first set of feature score, the first class, and the set of confusing class pairs for each datapoint in the first dataset. Each semantic class indicates a prediction accuracy of a dataset clustered in the semantic class. A classifier is trained based on the clustered first dataset, the first set of feature scores, and the set of semantic classes.

Abstract: According to an aspect of an embodiment, operations may include receiving a first data point associated with a real-time application and predicting a first class for the received first data point, by a Deep Neural Network (DNN) pre-trained for a classification task of the real-time application. The operations may further include extracting, from the DNN, a first set of features and a corresponding first set of weights, for the predicted first class. The extracted first set of features may be associated with a convolution layer of the DNN. The operations may further include determining, by a pre-trained classifier associated with the predicted first class, a confidence score for the predicted first class based on the extracted first set of features and the corresponding first set of weights. The operations may further include generating output information to indicate correctness of the predicted first class based on the determined confidence score.

Abstract: According to one or more embodiments, operations may include gathering a set of machine learning (ML) projects from one or more repositories of ML projects based on a filtering criteria. The operations may also include ensuring executability of ML pipelines in the set of ML projects. In addition, the operations may include identifying irrelevant portions of the ML pipelines in the set of ML projects. Moreover, the operations may include generating quality features for the set of ML projects. In addition, the operations may include generating diversity features for the set of ML projects. Moreover, the operations may include selecting a subset of ML projects from the set of ML projects based on the quality features and the diversity features. In addition, the operations may include storing the subset of ML projects in a corpus of ML projects that may be adapted for use in new ML projects.

Abstract: According to one or more embodiments, operations may include, extracting first features from existing machine learning (ML) projects and storing the first features in a corpus. In addition, the operations may include performing a first search on the corpus based on a first search query to generate a first ranked set of the existing ML projects. Moreover, the operations may include generating second features based on the first features of the first ranked set of the existing ML projects. Moreover, the operations may include performing a second search on the corpus based on a second search query to generate a second ranked set of the existing ML projects. In addition, the operations may include recommending a highest ranked existing ML project in the second ranked set of the existing ML projects as adaptable for use in a second ML project.

Abstract: According to one or more embodiments, operations may include normalizing machine learning (ML) pipelines of existing ML projects stored in a corpus of existing ML projects. The operations may also include extracting functional blocks from the normalized ML pipelines. In addition, the operations may include assigning a label to each of the functional blocks in the normalized ML pipelines. Moreover, the operations may include indexing each of the ML pipelines in the corpus based on the labels assigned to the functional blocks. In addition, the operations may include utilizing the labels assigned to the functional blocks in the corpus to generate a new pipeline to perform a new ML task on a new dataset of a new ML project.

Abstract: According to one or more embodiments, operations may include gathering a set of machine learning (ML) projects from one or more repositories of ML projects based on a filtering criteria. The operations may also include ensuring executability of ML pipelines in the set of ML projects. In addition, the operations may include identifying irrelevant portions of the ML pipelines in the set of ML projects. Moreover, the operations may include generating quality features for the set of ML projects. In addition, the operations may include generating diversity features for the set of ML projects. Moreover, the operations may include selecting a subset of ML projects from the set of ML projects based on the quality features and the diversity features. In addition, the operations may include storing the subset of ML projects in a corpus of ML projects that may be adapted for use in new ML projects.

Abstract: According to one or more embodiments, operations may include storing existing machine learning (ML) projects in a corpus. The operations may also include generating a search query for a new ML project based on a new dataset and a new ML task for the new ML project. In addition, the operations may include searching through the existing ML projects stored in the corpus, based on the search query, for a set of existing ML projects. Moreover, the operations may include merging the ML pipelines of the set of existing ML projects to generate a new ML pipeline for the new ML project. In addition, the operations may include adapting functional blocks of the new ML pipeline for the new ML project to enable the new ML pipeline to be executed to perform the new ML task on the new dataset.

Abstract: According to an aspect of an embodiment, operations may include receiving a first data point associated with a real-time application and predicting a first class for the received first data point, by a Deep Neural Network (DNN) pre-trained for a classification task of the real-time application. The operations may further include extracting, from the DNN, a first set of features and a corresponding first set of weights, for the predicted first class. The extracted first set of features may be associated with a convolution layer of the DNN. The operations may further include determining, by a pre-trained classifier associated with the predicted first class, a confidence score for the predicted first class based on the extracted first set of features and the corresponding first set of weights. The operations may further include generating output information to indicate correctness of the predicted first class based on the determined confidence score.

Abstract: A method of automated candidate repair patch generation may include synthesizing repair expressions based on project code. Concise repair expressions may be mined from a code database. A set of repair expressions may be generated based on the synthesized repair expressions and the mined repair expressions. The set of repair expressions may include further repair expressions further synthesized from the synthesized repair expressions and the mined repair expressions. Candidate patches may be generated based on a repair schema and the set of repair expressions.

Abstract: A method may include obtaining a deep neural network model and obtaining a first training data point and a second training data point for the deep neural network model during a first training epoch. The method may include determining a first robustness value of the first training data point and a second robustness value of the second training data point. The method may further include omitting augmenting the first training data point in response to the first robustness value satisfying a robustness threshold and augmenting the second training data point in response to the second robustness value failing to satisfy the robustness threshold. The method may also include training the deep neural network model on the first training data point and the augmented second training data point during the first training epoch.

Abstract: A method may include obtaining multiple lines of programming code of a program, and obtaining multiple test cases for testing the program, where each of the test cases includes an assertion upon which a result of a respective test case is based. The method may also include executing the program for each of the test cases, and identifying affected lines of programming code that influence the assertions. The method may additionally include calculating a risk score for at least one of the lines of programming code based on the affected lines of programming code and the assertion, the risk score indicative of a likelihood that the at least one of the lines of programming code includes a fault.

Abstract: A method of automated candidate repair patch generation may include synthesizing repair expressions based on project code. Concise repair expressions may be mined from a code database. A set of repair expressions may be generated based on the synthesized repair expressions and the mined repair expressions. The set of repair expressions may include further repair expressions further synthesized from the synthesized repair expressions and the mined repair expressions. Candidate patches may be generated based on a repair schema and the set of repair expressions.

Abstract: A method may include obtaining multiple lines of programming code of a program, and obtaining multiple test cases for testing the program, where each of the test cases includes an assertion upon which a result of a respective test case is based. The method may also include executing the program for each of the test cases, and identifying affected lines of programming code that influence the assertions. The method may additionally include calculating a risk score for at least one of the lines of programming code based on the affected lines of programming code and the assertion, the risk score indicative of a likelihood that the at least one of the lines of programming code includes a fault.

Abstract: According to an aspect of an embodiment, a method may include executing multiple tests with respect to code under test of a software program to perform multiple test executions. The method may further include identifying one or more passing tests and one or more failing tests of the test executions. In addition, the method may include determining an aggregated score for each statement based on two or more of: the passing tests and the failing tests; a semantic similarity between one or more statement tokens included in the respective statement and one or more report tokens included in an error report; and an amount of time that has passed from when the respective statement received a change. Moreover, the method may include identifying a particular statement of the plurality of statements as a fault location in the code under test based on the aggregated scores of the plurality of statements.

Abstract: Operations may include obtaining a first code snippet associated with a fault location of a fault of the software program. The operations may further include obtaining a second code snippet. In addition, the operations may include determining element similarity between first elements of the first code snippet and second elements of the second code snippet. Further, the operations may include generating, based on the determined element similarity, an element map that maps the first elements of the first code statement to the second elements of the second code statement. The operations may further include obtaining an abstract program modification as a repair candidate of the fault. In addition, the operations may include generating a first repair based on the abstract program modification and the element map. Moreover, the operations may include generating a second repair based on the abstract program modification and the element map.

Abstract: A method of evaluating the robustness of a Deep Neural Network (DNN) model. The method includes obtaining a set of training data-points correctly predicted by the DNN model and obtaining a set of realistic transformations of the set of training data-points correctly predicted by the DNN model, where the set of realistic transformations corresponding to additional data-points within a predetermined mathematical distance from each of a training data-point of the set of training data-points. The method also includes creating a robustness profile corresponding to whether the DNN model accurately predicts an outcome for the additional data-points of the set of realistic transformations and generating a robustness evaluation of the DNN model based on the robustness profile.

Abstract: Operations may include obtaining a first code snippet associated with a fault location of a fault of the software program. The operations may further include obtaining a second code snippet. In addition, the operations may include determining element similarity between first elements of the first code snippet and second elements of the second code snippet. Further, the operations may include generating, based on the determined element similarity, an element map that maps the first elements of the first code statement to the second elements of the second code statement. The operations may further include obtaining an abstract program modification as a repair candidate of the fault. In addition, the operations may include generating a first repair based on the abstract program modification and the element map. Moreover, the operations may include generating a second repair based on the abstract program modification and the element map.

Abstract: The disclosed method may include accessing features including feature information of one or more candidate target projects and of a subject project, in which the candidate target projects and the subject project are software programs. The method may include determining a similarity score between the feature information of each of the candidate target projects and the feature information of the subject project, in which a similarity score is determined for each feature of each of the candidate target projects. The method may include aggregating the similarity scores of the feature information of each feature in the candidate target projects to create an aggregate similarity score for each of the candidate target projects and generate a set of similar target projects. The method may include modifying the subject project by implementing recommended code, based on the similar target projects, in the subject project to repair a defect.