Abstract: Embodiments described herein provide a unified debugging framework that adapts a pretrained programming language model for line-level debugging and repair. Specifically, the debugging framework follow the logic of programmers on how to debug their code. For example, the debugging framework first determines whether or not a function is buggy. If it is buggy, the debugging framework localizes the problematic line and provide a patch (repair).

Abstract: Embodiments described herein provide systems and methods for providing zero-shot visual question answering. A first image and a first question relating to a visual content of the first image are received. One or more image captions relevant to the first question are determined using a visual-language neural model by determining portions of the first image relevant to the first question. Answer candidates are generated using the one or more image captions, answer candidates. Synthetic question-answer pairs are generated using synthetic questions generated using the answer candidates and the answer candidates. A prompt is generated by concatenating the synthetic question-answer pairs. A first answer to the first question is generated using a language network model using an input of the first question prepended with the prompt.

Abstract: A system uses a neural network based model to perform scene text recognition. The system achieves high accuracy of prediction of text from scenes based on a neural network architecture that uses double attention mechanism. The neural network based model includes a convolutional neural network component that outputs a set of visual features and an attention extractor neural network component that determines attention scores based on the visual features. The visual features and the attention scores are combined to generate mixed features that are provided as input to a character recognizer component that determines a second attention score and recognizes the characters based on the second attention score. The system trains the neural network based model by adjusting the neural network parameters to minimize a multi-class gradient harmonizing mechanism (GHM) loss. The multi-class GHM loss varies based on a level of difficulty of the sample.

Abstract: Embodiments described herein a code generation and understanding model that builds on a Transformer-based encoder-decoder framework. The code generation and understanding model is configured to derive generic representations for programming language (PL) and natural language (NL) in code domain via pre-training on unlabeled code corpus, and then to benefit many code-related downstream tasks with fine-tuning. Apart from the denoising sequence-to-sequence objectives widely adopted for pre-training on natural language, identifier tagging and prediction pre-training objective is adopted to enable the model to better leverage the crucial token type information from PL, which specifically are the identifiers assigned by developers.

Abstract: Embodiments described herein provide a parameter-efficient finetuning mechanism, referred to as “factor-tuning,” which first learns a compact representation of parameter changes with existing datasets on multiple domains, and then fine-tunes a small number of parameters (automatically extracted from the learned representation) on a new downstream task. In this way, the representation learned in the first step is shared across domains and transferred to new downstream tasks.

Abstract: Embodiments described herein provide label modular prompts for a text classification task. A label modular prompt generator may determine a set of class labels of interest from a set of possible class labels associated with an input text sequence. The label modular prompt generator may generate a plurality of label prompts based on the set of class labels of interest. A first class label and a sequence of soft tokens that are generated based on representations associated with the first class label are concatenated into a first label prompt. The soft tokens are tunable using a plurality of parameters of the label modular prompt generator. The label modular prompt generator may provide an input of the input text sequence prepended with the plurality of label prompts to a pretrained language model. The pretrained language model may generate a task output in response to the input text sequence.

Abstract: Embodiments described herein provide training a prompt generator for text classification. A first training dataset associated with a first plurality of class labels is received for a first training process. For a first instance of the first training dataset, a set of labels of interest is generated by sampling from a set of possible class labels including the first plurality of class labels. The prompt generator generates a first prompt based on the set of labels of interest. A pretrained language model generates a task output in response to an input of the first instance prepended with the first prompt. A loss objective is generated based on the task output and the set of labels of interest. Parameters of the prompt generator are updated based on the computed loss function via backpropagation while the pretrained language model is frozen.

Abstract: Embodiments described herein provide a zero-shot visual question answering (VQA) framework, which conjoins foundation network models with zero additional training. A first image and a question relating to the first image are received. The first image is divided into a plurality of image patches. A plurality of relevant image patches that are relevant to the question are determined, using a first neural network model, from the plurality of image patches. A plurality of image captions are generated, using a second neural network model, based on the plurality of relevant image patches. An answer to the question is generated based on the plurality of image captions.

Abstract: Systems and methods for providing a neural network system for time series forecasting are described. A time series dataset that includes datapoints at a plurality of timestamps in an observed space is received. The neural network system is trained using the time series dataset. The training the neural network includes: generating, using an encoder of the neural network system, one or more estimated latent variables of a latent space for the time series dataset; generating, using an auxiliary predictor of the neural network system, a first latent-space prediction result based on the one or more estimated latent variables; transforming, using a decoder of the neural network system, the first latent-space prediction result to a first observed-space prediction result; and updating parameters of the neural network system based on a loss based on the first observed-space prediction result.

Abstract: Some embodiments of the current disclosure disclose methods and systems for analyzing root causes of an incident disrupting information technology services such as cloud services. In some embodiments, a set of problem review board (PRB) documents including information about said incidents may be parsed using a natural language processing (NLP) neural model to extract structured PRB data from the unstructured investigative information contained in the PRB documents. The structured PRB data may include symptoms of the incident, root causes of the incident, resolutions of the incidents, etc., and a causal knowledge graph causally relating the symptoms, root causes, resolutions of the incidents may be generated.

Abstract: Systems and methods for providing a neural network system for time series forecasting are described. A time series dataset that includes datapoints at a plurality of timestamps in an observed space is received. A first state-space model of a dynamical system underlying the time series dataset is provided. The first state-space model includes a non-parametric latent transition model. One or more latent variables of a latent space for the time series dataset are determined using the neural network system based on the first state-space model. A first prediction result for the time series dataset is provided by the neural network system based on the estimated latent variables.

Abstract: Embodiments described herein provide a reinforcement learning based framework engaging pretrained language models (LMs) for program synthesis tasks. Specifically, the framework adopts a training strategy that optimizes pretrained LMs for program synthesis tasks in an actor-critic approach.

Abstract: Embodiments described herein provide a time-index model for forecasting time-series data. The architecture of the model takes a normalized time index as an input, uses a model, g_?, to produce a vector representation of the time-index, and uses a “ridge regressor” which takes the vector representation and provides an estimated value. The model may be trained on a time-series dataset. The ridge regressor is trained for a given g_? to reproduce a given lookback window. g_? is trained over time-indexes in a horizon window, such that g_? and the corresponding ridge regressor will accurately predict the data in the horizon window. Once g_? is sufficiently trained, the ridge regressor can be updated based on that final g_? over a lookback window comprising the time-indexes with the last known values. The final g_? together with the updated ridge regressor can be given time-indexes past the known values, thereby providing forecasted values.

Abstract: Embodiments described herein provide a reinforcement learning based framework engaging pretrained language models (LMs) for program synthesis tasks. Specifically, the framework adopts a training strategy that optimizes pretrained LMs for program synthesis tasks in an actor-critic approach.

Abstract: Systems and methods for automatic program repair using neural network models are described. After a first buggy code patch is received, a first representation of the first buggy code patch is generated using a retriever encoder of a patch retriever. The patch retriever retrieves, based on the first representation, a first bug-fix code pair from a first plurality of bug-fix code pairs. A first augmented buggy code patch is generated based on the first buggy code patch and the first bug-fix code pair. A patch generator generates a fixed code patch based on the first augmented buggy code patch.

Abstract: Embodiments described herein provide using a measure of distance between time-series data sequences referred to as optimal transport warping (OTW). Measuring the OTW distance between unbalanced sequences (sequences with different sums of their values) may be accomplished by including an unbalanced mass cost. The OTW computation may be performed using cumulative sums over local windows. Further, embodiments herein describe methods for dealing with time-series data with negative values. Sequences may be split into positive and negative components before determining the OTW distance. A smoothing function may also be applied to the OTW measurement allowing for a gradient to be calculated. The OTW distance may be used in machine learning tasks such as clustering and classification. An OTW measurement may also be used as an input layer to a neural network.

Abstract: Embodiments described herein provide a masked self-training (MaST) which is an unsupervised learning approach leveraging two complimentary sources of supervision: pseudo-labels and raw image pixels. Specifically, MaST jointly optimizes three objectives to finetune a pre-trained classification model on unlabeled images: (1) self-training objective to learn global task-specific class prediction; (2) masked image modeling objective to learn local pixel-level information; (3) global-local feature alignment objective to bridge the knowledge learned from the two sources of supervision.

Abstract: Embodiments described herein provide a soft prompt tuning technique referred to as the Vector quantized Input-contextualized Prompt (VIP). The VIP techniques has two integral properties i) instead of learning a fixed set of prompt tokens irrespective of the input, it generates a contextualized version of the soft prompts, conditional on the input text ii) it further passes the input-contextualized prompt tokens through a quantization network, inspired by Vector Quantized Transformers. The quantization network uses nearest neighbor search over a learnable codebook to train a discrete latent variable model over the prompt-space, thus generating quantized version of contextual prompt tokens. These quantized contextual prompt tokens are finally fed into the frozen language model along with the original input text.

Abstract: Embodiments described herein provide a soft prompt tuning technique referred to as the Vector quantized Input-contextualized Prompt (VIP). The VIP techniques has two integral properties i) instead of learning a fixed set of prompt tokens irrespective of the input, it generates a contextualized version of the soft prompts, conditional on the input text ii) it further passes the input-contextualized prompt tokens through a quantization network, inspired by Vector Quantized Transformers. The quantization network uses nearest neighbor search over a learnable codebook to train a discrete latent variable model over the prompt-space, thus generating quantized version of contextual prompt tokens. These quantized contextual prompt tokens are finally fed into the frozen language model along with the original input text.

Abstract: Embodiments described herein provide an Adapt-and-Adjust (A2) mechanism for multilingual speech recognition model that combines both adaptation and adjustment methods as an integrated end-to-end training to improve the models' generalization and mitigate the long-tailed issue. Specifically, a multilingual language model mBERT is utilized, and converted into an autoregressive transformer decoder. In addition, a cross-attention module is added to the encoder on top of the mBERT's self-attention layer in order to explore the acoustic space in addition to the text space. The joint training of the encoder and mBERT decoder can bridge the semantic gap between the speech and the text.