Abstract: A method, computer program product, and computer system. A generative large language model (LLM) receives a query of text and associated text data that includes N units of text for determining a context for the query. An embedding (Zquery) of the query and a vector Z including N embeddings is generated. The N elements of Z are distributed into C memory portions of the external memory unit. Each memory portion includes S memory locations, where C=CEILING (N/S). A process (a) ascertains a closest element in each memory portion that is closest to Zquery, and (b) distributes the ascertained closest elements into memory locations within a proper subset of the C memory portions, where (a) and (b) are recursively repeated until only a single closest element is ascertained from which a context is determined. A response to the query is generated based on the query and the context.

Abstract: A method, system and apparatus of ensembling, including inputting a set of models that predict different sets of attributes, determining a source set of attributes and a target set of attributes using a barycenter with an optimal transport metric, and determining a consensus among the set of models whose predictions are defined on the source set of attributes.

Abstract: A distilled machine learning model is produced via initializing a first model with initial weights. An input protein sequence is input into both the first model and a folding protein model, wherein the inputting to the first model generates logits, and wherein the inputting to the folding protein model generates one or more predictive metrics. The one or more predictive metrics are discretized into classes and a first cross-entropy loss is computed based on the logits and the classes. The first model is optimized based on the first cross-entropy loss so that the optimized first model is the distilled machine learning model and an additional machine learning model is trained, using the distilled machine learning model, to perform a downstream protein modeling task.

Abstract: A first language vector can be generated by performing a first linear projection on a partial amino acid sequence vector. A second language vector can be generated by performing natural language processing on the first language vector. A predicted amino acid sequence vector can be generated by performing a second linear projection on the second language vector. A complete amino acid sequence listing can be output based on the predicted amino acid sequence vector.

Abstract: Obtain access to a pretrained encoder-decoder language model. Using a dataset including a plurality of text-graph pairs, carry out first fine-tuning training on the pre-trained language model by minimizing cross-entropy loss. A text portion of each text-graph pair includes a list of text tokens and a graph portion of each text-graph pair includes a list of graph tokens. The first fine-tuning training results in an intermediate model. Carry out second fine-tuning training on the intermediate model, by reinforcement learning, to obtain a final model. Make the final model available for deployment.

Abstract: A knowledge graph is constructed as part of a multi-stage process using pretrained language models. Input text in a natural language format is received. In a first stage, a plurality of nodes is generated using a pretrained language model, where the nodes correspond to entities of the input text. In the second stage edges to interconnect the plurality of nodes are generated. The edges are generated responsive to generating each of the plurality of nodes.

Abstract: Systems, computer-implemented methods, and computer program products to facilitate a dual learning bridge between text and a knowledge graph are provided. According to an embodiment, a system can comprise a processor that executes computer executable components stored in memory. The computer executable components comprise a model component that employs a model to learn associations between text data and a knowledge graph. The computer executable components further comprise a translation component that uses the model to bidirectionally translate second text data and one or more knowledge graph paths based on the associations.

Abstract: A computing device receives a data X and Y, each having N samples. A function f(x,y) is defined to be a trainable neural network based on the data X and the data Y. A permuted version of the data Y is created. A loss mean is computed based on the trainable neural network f(x,y), the permuted version of the sample data Y, and a trainable scalar variable ?. A loss with respect to the scalar variable ? and the trainable neural network is minimized. Upon determining that the loss is at or below the predetermined threshold, estimating a mutual information (MI) between a test data XT and YT. If the estimated MI is above a predetermined threshold, the test data XT and YT is deemed to be dependent. Otherwise, it is deemed to be independent.

Abstract: A system and method for designing protein sequences conditioned on a specific target fold. The system is a transformer-based generative framework for modeling a complex sequence-structure relationship. To mitigate the heterogeneity between the sequence domain and the fold domain, a Fold-to-Sequence model jointly learns a sequence embedding using a transformer and a fold embedding from the density of secondary structural elements in 3D voxels. The joint sequence-fold representation through novel intra-domain and cross-domain losses with an intra-domain loss forcing two semantically similar (where the proteins should have the same fold(s)) samples from the same domain to be close to each other in a latent space, while a cross-domain loss forces two semantically similar samples in different domains to be closer. In an embodiment, the Fold-to-Sequence model performs design tasks that include low resolution structures, structures with region of missing residues, and NMR structural ensembles.

Abstract: Systems, computer-implemented methods, and computer program products to facilitate a dual learning bridge between text and a knowledge graph are provided. According to an embodiment, a system can comprise a processor that executes computer executable components stored in memory. The computer executable components comprise a model component that employs a model to learn associations between text data and a knowledge graph. The computer executable components further comprise a translation component that uses the model to bidirectionally translate second text data and one or more knowledge graph paths based on the associations.

Abstract: With at least one hardware processor, obtain data specifying: two trained neural network models; and alignment data. With the at least one hardware processor, carry out neuron alignment on the two trained neural network models using the alignment data to obtain two aligned models. With the at least one hardware processor, train a minimal loss curve between the two aligned models. With the at least one hardware processor, select a new model along the minimal loss curve that maximizes accuracy on adversarially perturbed data.

Abstract: A computing device receives a data X and Y, each having N samples. A function f(x,y) is defined to be a trainable neural network based on the data X and the data Y. A permuted version of the data Y is created. A loss mean is computed based on the trainable neural network f(x,y), the permuted version of the sample data Y, and a trainable scalar variable ?. A loss with respect to the scalar variable ? and the trainable neural network is minimized. Upon determining that the loss is at or below the predetermined threshold, estimating a mutual information (MI) between a test data XT and YT. If the estimated MI is above a predetermined threshold, the test data XT and YT is deemed to be dependent. Otherwise, it is deemed to be independent.

Abstract: A method, system and apparatus of ensembling, including inputting a set of models that predict different sets of attributes, determining a source set of attributes and a target set of attributes using a barycenter with an optimal transport metric, and determining a consensus among the set of models whose predictions are defined on the source set of attributes.

Abstract: An unsupervised text style transfer method, system, and computer program product include classifying a style of an input message, translating the input message into a second style, re-writing the input message into a second message having the second style, and distributing the second message in the second style.

Abstract: A computer-implemented method for controlling a manufacturing process. A non-limiting example of the computer-implemented method includes using a processor to perform discretization modeling of a continuous probability distribution to yield a prediction of a future probability distribution. Next, the method uses the processor to impose a smoothness condition on the predicted probability distribution. The method using the processor to perform a multi-step forecast of the probability distribution to create a predicted probability density function. The method uses the predicted probability density function as an input to a process control system and uses the processor to control a process using the predicted probability density function.