Abstract: A method for generating binding peptides presented by any given Major Histocompatibility Complex (MHC) protein is presented. The method includes, given a peptide and an MHC protein pair, enabling a Reinforcement Learning (RL) agent to interact with and exploit a peptide mutation environment by repeatedly mutating the peptide and observing an observation score of the peptide, learning to form a mutation policy, via a mutation policy network, to iteratively mutate amino acids of the peptide to obtain desired presentation scores, and generating, based on the desired presentation scores, qualified peptides and binding motifs of MHC Class I proteins.

Abstract: A method for generating binding peptides presented by any given Major Histocompatibility Complex (MHC) protein is presented. The method includes, given a peptide and an MHC protein pair, enabling a Reinforcement Learning (RL) agent to interact with and exploit a peptide mutation environment by repeatedly mutating the peptide and observing an observation score of the peptide, learning to form a mutation policy, via a mutation policy network, to iteratively mutate amino acids of the peptide to obtain desired presentation scores, and generating, based on the desired presentation scores, qualified peptides and binding motifs of MHC Class I proteins.

Abstract: Methods and systems for training a model include training an encoder in an unsupervised fashion based on a backward latent flow between a reference frame and a driving frame taken from a same video. A diffusion model is trained that generates a video sequence responsive to an input image and a text condition, using the trained encoder to determine a latent flow sequence and occlusion map sequence of a labeled training video.

Abstract: Methods and systems for image generation include generating a latent representation of an image, modifying the latent representation of the image based on a trained attribute classifier and a specified attribute input, and decoding the modified latent representation to generate an output image that matches the specified attribute input.

Abstract: A computer-implemented method for learning disentangled representations for T-cell receptors to improve immunotherapy is provided. The method includes optionally introducing a minimal number of mutations to a T-cell receptor (TCR) sequence to enable the TCR sequence to bind to a peptide, using a disentangled Wasserstein autoencoder to separate an embedding space of the TCR sequence into functional embeddings and structural embeddings, feeding the functional embeddings and the structural embeddings to a long short-term memory (LSTM) or transformer decoder, using an auxiliary classifier to predict a probability of a positive binding label from the functional embeddings and the peptide, and generating new TCR sequences with enhanced binding affinity for immunotherapy to target a particular virus or tumor.

Abstract: A system for binding peptide search for immunotherapy is presented. The system includes employing a deep neural network to predict a peptide presentation given Major Histocompatibility Complex allele sequences and peptide sequences, training a Variational Autoencoder (VAE) to reconstruct peptides by converting the peptide sequences into continuous embedding vectors, running a Monte Carlo Tree Search to generate a first set of positive peptide vaccine candidates, running a Bayesian Optimization search with the trained VAE and a Backpropagation search with the trained VAE to generate a second set of positive peptide vaccine candidates, using a sampling from a Position Weight Matrix (sPWM) to generate a third set of positive peptide vaccine candidates, screening and merging the first, second, and third sets of positive peptide vaccine candidates, and outputting qualified peptides for immunotherapy from the screened and merged sets of positive peptide vaccine candidates.

Abstract: A system for binding peptide search for immunotherapy is presented. The system includes employing a deep neural network to predict a peptide presentation given Major Histocompatibility Complex allele sequences and peptide sequences, training a Variational Autoencoder (VAE) to reconstruct peptides by converting the peptide sequences into continuous embedding vectors, running a Monte Carlo Tree Search to generate a first set of positive peptide vaccine candidates, running a Bayesian Optimization search with the trained VAE and a Backpropagation search with the trained VAE to generate a second set of positive peptide vaccine candidates, using a sampling from a Position Weight Matrix (sPWM) to generate a third set of positive peptide vaccine candidates, screening and merging the first, second, and third sets of positive peptide vaccine candidates, and outputting qualified peptides for immunotherapy from the screened and merged sets of positive peptide vaccine candidates.

Abstract: A system for binding peptide search for immunotherapy is presented. The system includes employing a deep neural network to predict a peptide presentation given Major Histocompatibility Complex allele sequences and peptide sequences, training a Variational Autoencoder (VAE) to reconstruct peptides by converting the peptide sequences into continuous embedding vectors, running a Monte Carlo Tree Search to generate a first set of positive peptide vaccine candidates, running a Bayesian Optimization search with the trained VAE and a Backpropagation search with the trained VAE to generate a second set of positive peptide vaccine candidates, using a sampling from a Position Weight Matrix (sPWM) to generate a third set of positive peptide vaccine candidates, screening and merging the first, second, and third sets of positive peptide vaccine candidates, and outputting qualified peptides for immunotherapy from the screened and merged sets of positive peptide vaccine candidates.

Abstract: A method for generating binding peptides presented by any given Major Histocompatibility Complex (MHC) protein is presented. The method includes, given a peptide and an MHC protein pair, enabling a Reinforcement Learning (RL) agent to interact with and exploit a peptide mutation environment by repeatedly mutating the peptide and observing an observation score of the peptide, learning to form a mutation policy, via a mutation policy network, to iteratively mutate amino acids of the peptide to obtain desired presentation scores, and generating, based on the desired presentation scores, qualified peptides and binding motifs of MHC Class I proteins.

Abstract: Methods and systems for video processing include extracting flow features and appearance features from frames of a video stream. The flow features are processed using a flow model that is trained on a first set of training data. An output of the flow model is processed using a sub-network that is trained on the first set of training data and a second set of domain-specific training data to generate a flow parameter. The appearance features are processed using an appearance model that is trained on the first set of training data and that further processes the appearance features using the flow parameter, to classify the frames of the video stream. An action is performed responsive to the classified frames.

Abstract: Methods and systems for video processing include enriching an input video feature from an input video frame set using a meta-action bank video sub-actions to generate enriched features. Reinforced image representation is performed using reinforcement learning to compare support image frames and query image frames and determine an importance of the input video frame. A classification is performed on the input video frame based on the importance and the enriched features to generate a label. An action is performed responsive to the generated label.

Abstract: Methods and systems for temporal action localization include processing a video stream to identify an action and a start time and a stop time for the action using a neural network model that separately processes information of appearance and motion modalities from the video stream using transformer branches that include a self-attention and a cross-attention between the appearance and motion modalities. An action is performed responsive to the identified action.

Abstract: Methods and systems for disentangled data generation include accessing a dataset including pairs, each formed from a given input text structure and a given style label for the input text structures. An encoder is trained to disentangle a sequential text input into disentangled representations, including a content embedding and a style embedding, based on a subset of the dataset, using an objective function that includes a regularization term that minimizes mutual information between the content embedding and the style embedding. A generator is trained to generate a text output that includes content from the style embedding, expressed in a style other than that represented by the style embedding of the text input.

Abstract: A computer-implemented method is provided for generating new binding peptides to Major Histocompatibility Complex (MHC) proteins. The method includes training, by a processor device, a Generative Adversarial Network GAN having a generator and a discriminator only on a set of binding peptide sequences given training data comprising the set of binding peptide sequences and a set of non-binding peptide sequences. A GAN training objective includes the discriminator being iteratively updated to distinguish generated peptide sequences from sampled binding peptide sequences as fake or real and the generator being iteratively updated to fool the discriminator. The training includes optimizing the GAN training objective while learning two projection vectors for a binding class with two cross-entropy losses. A first loss discriminating binding peptide sequences in the training data from non-binding peptide sequences in the training data.

Abstract: A computer-implemented method is provided for generating new binding peptides to Major Histocompatibility Complex (MHC) proteins. The method includes training, by a processor device, a Generative Adversarial Network GAN having a generator and a discriminator only on a set of binding peptide sequences given training data comprising the set of binding peptide sequences and a set of non-binding peptide sequences. A GAN training objective includes the discriminator being iteratively updated to distinguish generated peptide sequences from sampled binding peptide sequences as fake or real and the generator being iteratively updated to fool the discriminator. The training includes optimizing the GAN training objective while learning two projection vectors for a binding class with two cross-entropy losses. A first loss discriminating binding peptide sequences in the training data from non-binding peptide sequences in the training data.

Abstract: A computer-implemented method is provided for generating new binding peptides to Major Histocompatibility Complex (MHC) proteins. The method includes training, by a processor device, a Generative Adversarial Network GAN having a generator and a discriminator only on a set of binding peptide sequences given training data comprising the set of binding peptide sequences and a set of non-binding peptide sequences. A GAN training objective includes the discriminator being iteratively updated to distinguish generated peptide sequences from sampled binding peptide sequences as fake or real and the generator being iteratively updated to fool the discriminator. The training includes optimizing the GAN training objective while learning two projection vectors for a binding class with two cross-entropy losses. A first loss discriminating binding peptide sequences in the training data from non-binding peptide sequences in the training data.

Abstract: Methods and systems for peptide generation include training a peptide mutation policy neural network using reinforcement learning that includes a peptide presentation score as a reward. New peptides are generated using the peptide mutation policy. A binding motif of a major histocompatibility complex is calculated using the new peptides. Library peptides are screened in accordance with the binding motif.

Abstract: A method for implementing deep reinforcement learning with T-cell receptor (TCR) mutation policies to generate binding TCRs for immunotherapy includes extracting peptides to identify a virus or tumor cells, collecting a library of TCRs from patients, predicting interaction scores between the extracted peptides and the TCRs from the patients, developing a deep reinforcement learning framework with TCR mutation policies to generate TCRs with maximum binding scores, defining reward functions, outputting mutated TCRs, ranking the outputted TCRs to utilize top-ranked TCR candidates to target the virus or the tumor cells, and for each top-ranked TCR candidate, repeatedly identifying a set of self-peptides that the top-ranked TCR candidate binds to and further optimizing it greedily by maximizing a sum of its interaction scores with a given set of peptide antigens while minimizing a sum of its interaction scores with the set of self-peptides until stopping criteria of efficacy and safety are met.

Abstract: A method for using a multi-hop reasoning framework to perform multi-step compositional long-term reasoning is presented. The method includes extracting feature maps and frame-level representations from a video stream by using a convolutional neural network (CNN), performing object representation learning and detection, linking objects through time via tracking to generate object tracks and image feature tracks, feeding the object tracks and the image feature tracks to a multi-hop transformer that hops over frames in the video stream while concurrently attending to one or more of the objects in the video stream until the multi-hop transformer arrives at a correct answer, and employing video representation learning and recognition from the objects and image context to locate a target object within the video stream.

Abstract: A method for implementing deep reinforcement learning with T-cell receptor (TCR) mutation policies to generate binding TCRs recognizing target peptides for immunotherapy is presented. The method includes extracting peptides to identify a virus or tumor cells, collecting a library of TCRs from target patients, predicting, by a deep neural network, interaction scores between the extracted peptides and the TCRs from the target patients, developing a deep reinforcement learning (DRL) framework with TCR mutation policies to generate TCRs with maximum binding scores, defining reward functions based on a reconstruction-based score and a density estimation-based score, randomly sampling batches of TCRs and following a policy network to mutate the TCRs, outputting mutated TCRs, and ranking the outputted TCRs to utilize top-ranked TCR candidates to target the virus or the tumor cells for immunotherapy.