Abstract: An automated program repair tool utilizes a neural transformer model with attention to predict the contents of a bug repair in the context of source code having a bug of an identified bug type. The neural transformer model is trained on a large unsupervised corpus of source code using a span-masking denoising optimization objective, and fine-tuned on a large supervised dataset of triplets containing a bug-type annotation, software bug, and repair. The bug-type annotation is derived from an interprocedural static code analyzer. A bug type edit centroid is computed for each bug type and used in the inference decoding phase to generate the bug repair.

Abstract: A transfer learning system is used for the development of neural transformer models pertaining to software engineering tasks. The transfer learning system trains source code domain neural transformer models with attention in various configurations on a large corpus of unsupervised training dataset of source code programs and/or source code-related natural language text. A web service provides the trained models for use in developing a model that may be fine-tuned on a supervised training dataset associated with a software engineering task thereby generating a tool to perform the software engineering task.

Abstract: A neural transformer model with attention is trained to predict candidates to complete a line of source code with a zero-inference capability. The model is trained on an unsupervised training dataset that includes features from source code written in multiple programming languages. The features include a file-level context and a local context, where the file-level context includes a global context, a class context, a function context, and/or a method context for each class, function and/or method of the source code programs used in the training dataset. The local context includes method bodies, function bodies, and/or stand-alone code of main method routines. From these features, the model is able to learn to predict an ordered sequence of code elements that complete a line of source code in a programming language seen and not seen during training.

Abstract: A code completion tool uses a neural transformer model with attention to generate syntactically-correct candidates with holes to complete a partially-formed code snippet. The model is trained to predict the expansion of non-terminal symbols of the production rules of the underlying grammar of the code snippet without being constrained to a left-to-right expansion order. A hole is a non-terminal symbol of the grammar of a programming language that marks a position in a candidate where the code completion engine is not certain of the production rule that should be used to expand the non-terminal symbol. The hole allows the code completion engine to expand other non-terminal symbols in a candidate and allow the user to guide the expansion of the holes in a candidate.

Abstract: An automated system for translating source code written in one programming language into a different programming language utilizes a neural transformer with attention trained on semi-supervised data. The model is jointly pre-trained with a masked language model objective and an autoregressive objective on a large unsupervised source code corpus to learn to comprehend the syntactic structure and semantics of source code. The pre-trained model is then fine-tuned with a token-type prediction objective and an autoregressive objective on supervised translation tasks and data augmented tasks to learn to translate source code from one programming language into a different programming language.

Abstract: A neural transformer model with attention is trained to predict candidates to complete a line of source code with a zero-inference capability. The model is trained on an unsupervised training dataset that includes features from source code written in multiple programming languages. The features include a file-level context and a local context, where the file-level context includes a global context, a class context, a function context, and/or a method context for each class, function and/or method of the source code programs used in the training dataset. The local context includes method bodies, function bodies, and/or stand-alone code of main method routines. From these features, the model is able to learn to predict an ordered sequence of code elements that complete a line of source code in a programming language seen and not seen during training.

Abstract: A code completion tool uses a deep learning model to predict the likelihood of a method completing a method invocation. In one aspect, the deep learning model is a LSTM trained on features that represent the syntactic context of a method invocation derived from an abstract tree representation of the code fragment.

Abstract: An automated system for resolving program merges uses a sequence-to-sequence supervised machine learning model trained from developer-resolved merge conflicts to learn to predict a merge resolution to resolve a three-way program merge. The model utilizes an embedding of the merge tuple (A, B, O) which represents the program syntax, program semantics and the intent of the program inputs. The model uses a pointer mechanism to construct the resolved program in terms of the lines of source code found in the input programs.

Abstract: An automated system for resolving program merges uses neural transformers with attention. In one aspect, a neural encoder transformer model is trained from developer-resolved merge conflicts to learn to predict a resolution strategy that aids a developer in constructing a merged program. In a second aspect, a neural decoder transformer model is trained on the syntax and semantics of different source code programming languages to predict a merge resolution consisting of interleaved lines of source code from programs A, B, or O, where programs A and B contain changes to code base O.

Abstract: A code completion tool uses a neural transformer model to generate candidate sequences to complete a line of source code. The neural transformer model is trained using a conditional language modeling objective on a large unsupervised dataset that includes source code programs written in several different programming languages. The neural transformer model is used within a beam search that predicts the most likely candidate sequences for a code snippet under development.

Abstract: A code completion system uses neural components to rank the unordered list of code completion candidates generated from an existing static analyzer. The candidates represent the next sequence of tokens likely to complete a partially-formed program element as a developer is typing in a software development tool. A re-ranking component generates a ranked order of the candidates based on a context embedding of the code context and candidate embeddings of the candidates, where both embeddings are based a common token encoding.

Abstract: A unit test generation system employs a neural transformer model with attention to generate candidate unit test sequences given a focal method of a programming language. The neural transformer model is pre-trained with source code programs and natural language text and fine-tuned with mapped test case pairs. A mapped test case pair includes a focal method and a unit test case for the focal method. In this manner, the neural transformer model is trained to learn the semantics and statistical properties of a natural language, the syntax of a programming language and the relationships between the code elements of the programming language and the syntax of a unit test case.

Abstract: An assert statement generator employs a neural transformer model with attention to generate candidate assert statements for a unit test method that tests a focal method. The neural transformer model is pre-trained with source code programs and natural language text and fine-tuned with test-assert triplets. A test-assert triplet includes a source code snippet that includes: (1) a unit test method with an assert placeholder; (2) the focal method; and (3) a corresponding assert statement. In this manner, the neural transformer model is trained to learn the semantics and statistical properties of a natural language, the syntax of a programming language, and the relationships between the code elements of the programming language and the syntax of an assert statement.

Abstract: A code completion tool uses a neural transformer model to generate candidate sequences to complete a line of source code. The neural transformer model is trained using a conditional language modeling objective on a large unsupervised dataset that includes source code programs written in several different programming languages. The neural transformer model is used within a beam search that predicts the most likely candidate sequences for a code snippet under development.

Abstract: A transfer learning system is used for the development of neural transformer models pertaining to software engineering tasks. The transfer learning system trains source code domain neural transformer models with attention in various configurations on a large corpus of unsupervised training dataset of source code programs and/or source code-related natural language text. A web service provides the trained models for use in developing a model that may be fine-tuned on a supervised training dataset associated with a software engineering task thereby generating a tool to perform the software engineering task.

Abstract: A code completion tool uses a neural transformer model with attention to generate code documentation for a method in a particular code documentation style. The neural transformer model is trained with source code programs and natural language text. The neural transformer model is pre-trained to learn the meaning of a method name, its corresponding method parameters and types from a large corpus of unsupervised dataset of source code methods. The neural transformer model is then fine-tuned on translation tasks where the model leans to translate a method signature/method body into a docstring of particular code documentation style.

Abstract: An automated program repair tool utilizes a neural transformer model with attention to predict the contents of a bug repair in the context of source code having a bug of an identified bug type. The neural transformer model is trained on a large unsupervised corpus of source code using a span-masking denoising optimization objective, and fine-tuned on a large supervised dataset of triplets containing a bug-type annotation, software bug, and repair. The bug-type annotation is derived from an interprocedural static code analyzer. A bug type edit centroid is computed for each bug type and used in the inference decoding phase to generate the bug repair.

Abstract: A code completion tool uses a neural transformer model with attention to generate candidate sequences to complete a method body of a method signature. The neural transformer model is trained with source code programs and natural language text. The neural transformer model learns the meaning of a method name, its corresponding method parameters and types from a large corpus of unsupervised dataset of source code methods and a supervised dataset of tasks including source code constructs in combination with natural language docstrings to infer a candidate sequence of subtokens that represent a method body for a particular method signature.

Abstract: A code completion system uses neural components to rank the unordered list of code completion candidates generated from an existing static analyzer. The candidates represent the next sequence of tokens likely to complete a partially-formed program element as a developer is typing in a software development tool. A re-ranking component generates a ranked order of the candidates based on a context embedding of the code context and candidate embeddings of the candidates, where both embeddings are based a common token encoding.

Abstract: A code completion tool uses a deep learning model to predict the likelihood of a method completing a method invocation. In one aspect, the deep learning model is a LSTM trained on features that represent the syntactic context of a method invocation derived from an abstract tree representation of the code fragment.