Abstract: Systems and methods for processing video are provided. The method includes receiving a text-based description of active scenes and representing the text-based description as a word embedding matrix. The method includes using a text encoder implemented by neural network to output frame level textual representation and video level representation of the word embedding matrix. The method also includes generating, by a shared generator, frame by frame video based on the frame level textual representation, the video level representation and noise vectors. A frame level and a video level convolutional filter of a video discriminator are generated to classify frames and video of the frame by frame video as true or false. The method also includes training a conditional video generator that includes the text encoder, the video discriminator, and the shared generator in a generative adversarial network to convergence.

Abstract: A method is provided for peptide-based vaccine generation. The method receives a dataset of positive and negative binding peptide sequences. The method pre-trains a set of peptide binding property predictors on the dataset to generate training data. The method trains a Wasserstein Generative Adversarial Network (WGAN) only on the positive binding peptide sequences, in which a discriminator of the WGAN is updated to distinguish generated peptide sequences from sampled positive peptide sequences from the training data, and a generator of the WGAN is updated to fool the discriminator. The method trains the WGAN only on the positive binding peptide sequences while simultaneously updating the generator to minimize a kernel Maximum Mean Discrepancy (MMD) loss between the generated peptide sequences and the sampled peptide sequences and maximize prediction accuracies of a set of pre-trained peptide binding property predictors with parameters of the set of pre-trained peptide binding property predictors being fixed.

Abstract: A method is provided for visual inspection. The method includes learning, by a processor, group disentangled visual feature embedding vectors of input images. The input images include defective objects and defect-free objects. The method further includes generating, by the processor using a weight generation network, classification weights from visual features and semantic descriptions. Both the visual features and the semantic descriptions are for predicting defective and defect-free labels. The method also includes calculating, by the processor, a cosine similarity score between the classification weights and the group disentangled visual feature embedding vectors. The method additionally includes episodically training, by the processor, the weight generation network on the input images to update parameters of the weight generation network.

Abstract: A computer-implemented method and system are provided for training a model for New Class Categorization (NCC) of a test image. The method includes decoupling, by a hardware processor, a feature extraction part from a classifier part of a deep classification model by reparametrizing learnable weight variables of the classifier part as a combination of learnable variables of the feature extraction part and of a classification weight generator of the classifier part. The method further includes training, by the hardware processor, the deep classification model to obtain a trained deep classification model by (i) learning the feature extraction part as a multiclass classification task, and (ii) episodically training the classifier part by learning a classification weight generator which outputs classification weights given a training image.

Abstract: A false alarm reduction system and method are provided for reducing false alarms in an automatic defect detection system. The false alarm reduction system includes a defect detection system, generating a list of image boxes marking detected potential defects in an input image. The false alarm reduction system further includes a feature extractor, transforming each of the image boxes in the list into a respective set of numerical features. The false alarm reduction system also includes a classifier, computing as a classification outcome for the each of the image boxes whether the detected potential defect is a true defect or a false alarm responsive to the respective set of numerical features for each of the image boxes.

Abstract: A context-aware attention-based neural network is provided for answering an input question given a set of purportedly supporting statements for the input question. The neural network includes a processing element. The processing element is configured to calculate a question representation for the input question, based on word annotations and word-level attentions calculated for the input question. The processing element is further configured to calculate a sentence representation for each of the purportedly supporting statements, based on word annotations and word-level attentions calculated for each of the purportedly supporting statements. The processing element is also configured to calculate a context representation for the set of purportedly supporting statements with respect to the sentence representation for each of the purportedly supporting statements.

Abstract: A computer-implemented method is provided for disentangled data generation. The method includes accessing, by a bidirectional Long Short-Term Memory (LSTM) with a multi-head attention mechanism, a dataset including a plurality of pairs each formed from a given one of a plurality of input text structures and given one of a plurality of style labels for the plurality of input text structures. The method further includes training the bidirectional LSTM as an encoder to disentangle a sequential text input into disentangled representations comprising a content embedding and a style embedding based on a subset of the dataset. The method also includes training a unidirectional LSTM as a decoder to generate a next text structure prediction for the sequential text input based on previously generated text structure information and a current word, from a disentangled representation with the content embedding and the style embedding.

Abstract: Methods and systems for disentangled data generation include accessing a dataset including pairs, each formed from a given input text structure and a given style label for the input text structures. An encoder is trained to disentangle a sequential text input into disentangled representations, including a content embedding and a style embedding, based on a subset of the dataset, using an objective function that includes a regularization term that minimizes mutual information between the content embedding and the style embedding. A generator is trained to generate a text output that includes content from the style embedding, expressed in a style other than that represented by the style embedding of the text input.

Abstract: A computer-implemented method is provided for disentangled data generation. The method includes accessing, by a variational autoencoder, a plurality of supervision signals. The method further includes accessing, by the variational autoencoder, a plurality of auxiliary tasks that utilize the supervision signals as reward signals to learn a disentangled representation. The method also includes training the variational autoencoder to disentangle a sequential data input into a time-invariant factor and a time-varying factor using a self-supervised training approach which is based on outputs of the auxiliary tasks obtained by using the supervision signals to accomplish the plurality of auxiliary tasks.

Abstract: A surveillance system is provided that includes an image capture device configured to capture a video sequence of a target area that includes objects and is formed from a set of image frames. The system further includes a processor configured to apply a C3D to the image frames to obtain therefor (i) intermediate feature representations across L convolutional layers and (ii) top-layer features. The processor is further configured to produce a first word of a caption for the sequence by applying the top-layer features to a LSTM. The processor is further configured to produce subsequent words of the caption by (i) dynamically performing spatiotemporal attention and layer attention using the intermediate feature representations to form a context vector, and (ii) applying the LSTM to the context vector, a previous word of the caption, and a hidden state of the LSTM. The system includes a display device for displaying the caption.

Abstract: Methods and systems for detecting and correcting anomalous inputs include training a neural network to embed high-dimensional input data into a low-dimensional space with an embedding that preserves neighbor relationships. Input data items are embedded into the low-dimensional space to form respective low-dimensional codes. An anomaly is determined among the high-dimensional input data based on the low-dimensional codes. The anomaly is corrected.

Abstract: A false alarm reduction system is provided that includes a processor cropping each input image at randomly chosen positions to form cropped images of a same size at different scales in different contexts. The system further includes a CONDA-GMM, having a first and a second conditional deep autoencoder for respectively (i) taking each cropped image without a respective center block as input for measuring a discrepancy between a reconstructed and a target center block, and (ii) taking an entirety of cropped images with the target center block. The CONDA-GMM constructs density estimates based on reconstruction error features and low-dimensional embedding representations derived from image encodings. The processor determines an anomaly existence based on a prediction of a likelihood of the anomaly existing in a framework of a CGMM, given the context being a representation of the cropped image with the center block removed and having a discrepancy above a threshold.

Abstract: A computer-implemented method, computer program product, and computer processing system are provided for word embedding. The method includes receiving, by a processor device, a word embedding matrix. The method further includes generating, by a processor device, an average pooling vector and a max pooling vector, based on the word embedding matrix. The method also includes generating, by the processor device, a prediction by applying a Multi-Layer Perceptron (MLP) to the average pooling vector and the max pooling vector.

Abstract: Systems and methods for detecting adversarial examples are provided. The method includes generating encoder direct output by projecting, via an encoder, input data items to a low-dimensional embedding vector of reduced dimensionality with respect to the one or more input data items to form a low-dimensional embedding space. The method includes regularizing the low-dimensional embedding space via a training procedure such that the input data items produce embedding space vectors whose global distribution is expected to follow a simple prior distribution. The method also includes identifying whether each of the input data items is an adversarial or unnatural input. The method further includes classifying, during the training procedure, those input data items which have not been identified as adversarial or unnatural into one of multiple classes.

Abstract: A system for electronic message classification and delivery using a neural network architecture includes one or more computing devices associated with one or more users, and at least one computer processing system in communication with one or more computing devices over at least one network. The at least one computer processing system includes at least one processor operatively coupled to a memory device and configured to execute program code stored on the memory device to receive one or more inputs associated with one or more e-mails corresponding to the one or more users across the at least one network, classify the one or more e-mails by performing natural language processing based on one or more sets of filters conditioned on respective ones of the one or more inputs, and permit the one or more users access to the one or more classified e-mails via the one or more computing devices.

Abstract: A computer-implemented method and system are provided for training a model for New Class Categorization (NCC) of a test image. The method includes decoupling, by a hardware processor, a feature extraction part from a classifier part of a deep classification model by reparametrizing learnable weight variables of the classifier part as a combination of learnable variables of the feature extraction part and of a classification weight generator of the classifier part. The method further includes training, by the hardware processor, the deep classification model to obtain a trained deep classification model by (i) learning the feature extraction part as a multiclass classification task, and (ii) episodically training the classifier part by learning a classification weight generator which outputs classification weights given a training image.

Abstract: Systems and methods for processing video are provided. The method includes receiving a text-based description of active scenes and representing the text-based description as a word embedding matrix. The method includes using a text encoder implemented by neural network to output frame level textual representation and video level representation of the word embedding matrix. The method also includes generating, by a shared generator, frame by frame video based on the frame level textual representation, the video level representation and noise vectors. A frame level and a video level convolutional filter of a video discriminator are generated to classify frames and video of the frame by frame video as true or false. The method also includes training a conditional video generator that includes the text encoder, the video discriminator, and the shared generator in a generative adversarial network to convergence.

Abstract: A method is provided for visual inspection. The method includes learning, by a processor, group disentangled visual feature embedding vectors of input images. The input images include defective objects and defect-free objects. The method further includes generating, by the processor using a weight generation network, classification weights from visual features and semantic descriptions. Both the visual features and the semantic descriptions are for predicting defective and defect-free labels. The method also includes calculating, by the processor, a cosine similarity score between the classification weights and the group disentangled visual feature embedding vectors. The method additionally includes episodically training, by the processor, the weight generation network on the input images to update parameters of the weight generation network.

Abstract: Aspects of the present disclosure describe techniques for training a convolutional neural network using an inconsistent stochastic gradient descent (ISGD) algorithm. Training effort for training batches used by the ISGD algorithm are dynamically adjusted according to a determined loss for a given training batch which are classified into two sub states—well-trained or under-trained. The ISGD algorithm provides more iterations for under-trained batches while reducing iterations for well-trained ones.

Abstract: Methods and systems for training a neural network include sampling multiple local sub-networks from a global neural network. The local sub-networks include a subset of neurons from each layer of the global neural network. The plurality of local sub-networks are trained at respective local processing devices to produce trained local parameters. The trained local parameters from each local sub-network are averaged to produce trained global parameters.