Abstract: Dialogue summarization is challenging due to its multi-speaker standpoints, casual spoken language style, and limited labelled data. The embodiments are directed to a coarse-to-fine dialogue summarization model that improves abstractive dialogue summarization quality and enables granular controllability. A summary draft that includes key words for turns in a dialogue conversation history is created. The summary draft includes pseudo-labelled interrogative pronoun categories and noisy key phrases. The dialogue conversation history is divided into segments. A generate language model is trained to generate a segment summary for each dialogue segment using a portion of the summary draft that corresponds to at least one dialogue turn in the dialogue segment. A dialogue summary is generated using the generative language model trained using the summary draft.

Abstract: A conversation engine performs conversations with users using chatbots customized for performing a set of tasks that can be performed using an online system. The conversation engine loads a chatbot configuration that specifies the behavior of a chatbot including the tasks that can be performed by the chatbot, the types of entities relevant to each task, and so on. The conversation may be voice based and use natural language. The conversation engine may load different chatbot configurations to implement different chatbots. The conversation engine receives a conversation engine configuration that specifies the behavior of the conversation engine across chatbots. The system may be a multi-tenant system that allows customization of the chatbots for each tenant.

Abstract: A conversation engine performs conversations with users using chatbots customized for performing a set of tasks that can be performed using an online system. The conversation engine loads a chatbot configuration that specifies the behavior of a chatbot including the tasks that can be performed by the chatbot, the types of entities relevant to each task, and so on. The conversation may be voice based and use natural language. The conversation engine may load different chatbot configurations to implement different chatbots. The conversation engine receives a conversation engine configuration that specifies the behavior of the conversation engine across chatbots. The system may be a multi-tenant system that allows customization of the chatbots for each tenant.

Abstract: Systems and methods for unifying question answering and text classification via span extraction include a preprocessor for preparing a source text and an auxiliary text based on a task type of a natural language processing task, an encoder for receiving the source text and the auxiliary text from the preprocessor and generating an encoded representation of a combination of the source text and the auxiliary text, and a span-extractive decoder for receiving the encoded representation and identifying a span of text within the source text that is a result of the NLP task. The task type is one of entailment, classification, or regression. In some embodiments, the source text includes one or more of text received as input when the task type is entailment, a list of classifications when the task type is entailment or classification, or a list of similarity options when the task type is regression.

Abstract: The technology disclosed provides a so-called “joint many-task neural network model” to solve a variety of increasingly complex natural language processing (NLP) tasks using growing depth of layers in a single end-to-end model. The model is successively trained by considering linguistic hierarchies, directly connecting word representations to all model layers, explicitly using predictions in lower tasks, and applying a so-called “successive regularization” technique to prevent catastrophic forgetting. Three examples of lower level model layers are part-of-speech (POS) tagging layer, chunking layer, and dependency parsing layer. Two examples of higher level model layers are semantic relatedness layer and textual entailment layer. The model achieves the state-of-the-art results on chunking, dependency parsing, semantic relatedness and textual entailment.

Abstract: Computing systems may support image classification and image detection services, and these services may utilize object detection/image classification machine learning models. The described techniques provide for normalization of confidence scores corresponding to manipulated target images and for non-max suppression within the range of confidence scores for manipulated images. In one example, the techniques provide for generating different scales of a test image, and the system performs normalization of confidence scores corresponding to each scaled image and non-max suppression per scaled image These techniques may be used to provide more accurate image detection (e.g., object detection and/or image classification) and may be used with models that are not trained on modified image sets. The model may be trained on a standard (e.g. non-manipulated) image set but used with manipulated target images and the described techniques to provide accurate object detection.

Abstract: Approaches for interpretable counting for visual question answering include a digital image processor, a language processor, and a counter. The digital image processor identifies objects in an image, maps the identified objects into an embedding space, generates bounding boxes for each of the identified objects, and outputs the embedded objects paired with their bounding boxes. The language processor embeds a question into the embedding space. The scorer determines scores for the identified objects. Each respective score determines how well a corresponding one of the identified objects is responsive to the question. The counter determines a count of the objects in the digital image that are responsive to the question based on the scores. The count and a corresponding bounding box for each object included in the count are output. In some embodiments, the counter determines the count interactively based on interactions between counted and uncounted objects.

Abstract: Embodiments described herein combine both masked reconstruction and predictive coding. Specifically, unlike contrastive learning, the mutual information between past states and future states are directly estimated. The context information can also be directly captured via shifted masked reconstruction—unlike standard masked reconstruction, the target reconstructed observations are shifted slightly towards the future to incorporate more predictability. The estimated mutual information and shifted masked reconstruction loss can then be combined as the loss function to update the neural model.

Abstract: Embodiments described herein provide an attentive network framework that models dynamic attributes with item and feature interactions. Specifically, the attentive network framework first encodes basket item sequences and dynamic attribute sequences with time-aware padding and time/month encoding to capture the seasonal patterns (e.g. in app recommendation, outdoor activities apps are more suitable for summer time while indoor activity apps are better for winter). Then the attentive network framework applies time-level attention modules on basket items' sequences and dynamic user attributes' sequences to capture basket items to basket items and attributes to attributes temporal sequential patterns. After that, an intra-basket attentive module is used on items in each basket to capture the correlation information among items.

Abstract: Embodiments described herein provide natural language processing (NLP) systems and methods that utilize energy-based models (EBMs) to compute an exponentially-weighted energy-like term in the loss function to train an NLP classifier. Specifically, noise contrastive estimation (NCE) procedures are applied together with the EBM-based loss objectives for training the NLPs.

Abstract: A system performs conversations with users using chatbots customized for performing a set of tasks. The system may be a multi-tenant system that allows customization of the chatbots for each tenant. The system processes sentences that may include negation or coreferences. The system determines a confidence score for an input sentence using an intent detection model, for example, a neural network. The system modifies the sentence to generate a modified sentence, for example, by removing a negation or by replacing a pronoun with an entity. The system generates a confidence score for the modified sentence using the intent detection model. The system determines the intent of the sentence based on the confidence scores of the sentence and the modified sentence. The system performs tasks based on the determined intent and performs conversations with users based on the tasks.

Abstract: A system performs conversations with users using chatbots customized for performing a set of tasks. The system may be a multi-tenant system that allows customization of the chatbots for each tenant. The system receives a task configuration that maps tasks to entity types and an entity configuration that specifies methods for determining entities of a particular entity type. The system receives a user utterance and determines the intent of the user using an intent detection model, for example, a neural network. The intent represents a task that the user is requesting. The system determines one or more entities corresponding to the task. The system performs tasks based on the determined intent and the entities and performs conversations with users based on the tasks.

Abstract: The system has a form analysis module that receives an image of a form into which values have been filled for the possible fields of information on the form, such as first name, address, age, and the like. By using a library of form templates, a form analysis module allows both flexibility of form processing and simplicity for the user. That is, the techniques used by the form analysis module allow the processing of any form image for which the library has a form template example. The form image need not precisely match any form template, but rather may be scaled or shifted relative to a corresponding template. Additionally, the user need only provide the form image itself, without providing any additional exemplars, metadata for training, or the like.

Abstract: A computer-implemented method for dual sequence inference using a neural network model includes generating a codependent representation based on a first input representation of a first sequence and a second input representation of a second sequence using an encoder of the neural network model and generating an inference based on the codependent representation using a decoder of the neural network model. The neural network model includes a plurality of model parameters learned according to a machine learning process. The encoder includes a plurality of coattention layers arranged sequentially, each coattention layer being configured to receive a pair of layer input representations and generate one or more summary representations, and an output layer configured to receive the one or more summary representations from a last layer among the plurality of coattention layers and generate the codependent representation.

Abstract: The technology disclosed presents a novel spatial attention model that uses current hidden state information of a decoder long short-term memory (LSTM) to guide attention and to extract spatial image features for use in image captioning. The technology disclosed also presents a novel adaptive attention model for image captioning that mixes visual information from a convolutional neural network (CNN) and linguistic information from an LSTM. At each timestep, the adaptive attention model automatically decides how heavily to rely on the image, as opposed to the linguistic model, to emit the next caption word. The technology disclosed further adds a new auxiliary sentinel gate to an LSTM architecture and produces a sentinel LSTM (Sn-LSTM). The sentinel gate produces a visual sentinel at each timestep, which is an additional representation, derived from the LSTM's memory, of long and short term visual and linguistic information.

Abstract: Embodiments described herein provide safe policy improvement (SPI) in a batch reinforcement learning framework for a task-oriented dialogue. Specifically, a batch reinforcement learning framework for dialogue policy learning is provided, which improves the performance of the dialogue and learns to shape a reward that reasons the invention behind human response rather than just imitating the human demonstration.

Abstract: Computing systems may support image classification and image detection services, and these services may utilize object detection/image classification machine learning models. The described techniques provide for normalization of confidence scores corresponding to manipulated target images and for non-max suppression within the range of confidence scores for manipulated images. In one example, the techniques provide for generating different scales of a test image, and the system performs normalization of confidence scores corresponding to each scaled image and non-max suppression per scaled image These techniques may be used to provide more accurate image detection (e.g., object detection and/or image classification) and may be used with models that are not trained on modified image sets. The model may be trained on a standard (e.g. non-manipulated) image set but used with manipulated target images and the described techniques to provide accurate object detection.

Abstract: Embodiments described herein provide a two-stage online detection of action start system including a classification module and a localization module. The classification module generates a set of action scores corresponding to a first video frame from the video, based on the first video frame and video frames before the first video frames in the video. Each action score indicating a respective probability that the first video frame contains a respective action class. The localization module is coupled to the classification module for receiving the set of action scores from the classification module and generating an action-agnostic start probability that the first video frame contains an action start.

Abstract: A natural language processing system that includes a sentence selector and a question answering module. The sentence selector receives a question and sentences that are associated with a context. For a question and each sentence, the sentence selector determines a score. A score represents whether the question is answerable with the sentence. Sentence selector then generates a minimum set of sentences from the scores associated with the question and sentences. The question answering module generates an answer for the question from the minimum set of sentences.

Abstract: The technology disclosed provides a so-called “joint many-task neural network model” to solve a variety of increasingly complex natural language processing (NLP) tasks using growing depth of layers in a single end-to-end model. The model is successively trained by considering linguistic hierarchies, directly connecting word representations to all model layers, explicitly using predictions in lower tasks, and applying a so-called “successive regularization” technique to prevent catastrophic forgetting. Three examples of lower level model layers are part-of-speech (POS) tagging layer, chunking layer, and dependency parsing layer. Two examples of higher level model layers are semantic relatedness layer and textual entailment layer. The model achieves the state-of-the-art results on chunking, dependency parsing, semantic relatedness and textual entailment.