Abstract: Evaluation data sets may be programmatically generated for code generation models. An evaluation data set is obtained that includes items that correspond to different evaluation tests for a code generation system. The individual items of the evaluation data set maybe converted, including the conversion of a function signature for the items, the test statements for the items and using a code generation system to generate the body of the function.

Abstract: Pre-fix matching may constrain the generation of next token predictions. Input text to perform a next token prediction may be received. Multiple tokens may be determined from the input text, including a partial token. From possible tokens, one or more matching possible tokens with the partial token may be identified. Next token predictions may then be filtered using the identified possible tokens in order to ensure that the partial token is matched.

Abstract: Random token segmentation may be implemented for next token prediction. Text data may be received for training a machine learning model to predict a next token given input text tokens. Multiple tokens may be determined from the text data. Different ones of the multiple token may be randomly segmented in to sub-tokens. The machine learning model may then be trained using the multiple tokens including the respective sub-tokens as a training data set.

Abstract: Code completion suggestions may be proactively obtained and validated. An event that triggers obtaining a code completion suggestion for inclusion in a code file being edited using an integrated development environment may be detected. The code completion suggestion may be obtained. The characters of the code completion suggestion may be compared with characters added to the code file after the detection of the event that triggered obtaining the code completion suggestion to determine whether the code completion suggestion is valid. A valid code completion suggestion may then be displayed.

Abstract: Techniques for ontology linking of unstructured text as a service are described. A service may receive a request to link unstructured text to a standardized ontology, and the service may segment and tokenize the unstructured text and send the result to multiple services implementing multiple deep machine learning models trained to identify particular entities and one or more relationships between entities. The service may perform a search of the standardized ontology to identify a set of similar candidates from the standardized ontology for the detected entities and the one or more relationships, and then rank the set of similar candidates from the standardized ontology according to their similarity to the detected entities within the unstructured text. The output from the service may include a result identifying a highest ranked candidate of the set of similar candidates from the standardized ontology for the detected entities within the unstructured text.

Abstract: Techniques for entity and relationship detect from unstructured text as a service are described. A service may receive a request to identify entities within a provided unstructured text element, and the service may segment and tokenize the unstructured text and send the result to multiple services implementing multiple deep machine learning models trained to identify particular entities. The service may send additional requests to an additional service or services implementing additional deep machine learning models to identify relationships between detected attributes and ones of the detected entities. The outputs from all services can be analyzed and consolidated into a single result that identifies the entities, any attributes of the entities, and confidence scores indicating the confidence in each detected entity.

Abstract: The present disclosure is directed to optimizing transfer learning for neural networks by creating a dynamic transfer network configuration through gated architecture. In some embodiments, transfer learning implements multiple parameter sharing schemes across a source task and a target task. The gating architecture can learn the optimal parameter sharing schemes as the neural network is trained. In some embodiments, the system can be used in named entity recognition applications where the training data is limited.