Abstract: One aspect of systems and methods for data correction includes identifying a false label from among predicted labels corresponding to different parts of an input sample, wherein the predicted labels are generated by a neural network trained based on a training set comprising training samples and training labels corresponding to parts of the training samples; computing an influence of each of the training labels on the false label by approximating a change in a conditional loss for the neural network corresponding to each of the training labels; identifying a part of a training sample of the training samples and a corresponding source label from among the training labels based on the computed influence; and modifying the training set based on the identified part of the training sample and the corresponding source label to obtain a corrected training set.

Abstract: The present disclosure describes a model for large scale color prediction of objects identified in images. Embodiments of the present disclosure include an object detection network, an attention network, and a color classification network. The object detection network generates object features for an object in an image and may include a convolutional neural network (CNN), region proposal network, or a ResNet. The attention network generates an attention vector for the object based on the object features, wherein the attention network takes a query vector based on the object features, and a plurality of key vector and a plurality of value vectors corresponding to a plurality of colors as input. The color classification network generates a color attribute vector based on the attention vector, wherein the color attribute vector indicates a probability of the object including each of the plurality of colors.

Abstract: A group captioning system includes computing hardware, software, and/or firmware components in support of the enhanced group captioning contemplated herein. In operation, the system generates a target embedding for a group of target images, as well as a reference embedding for a group of reference images. The system identifies information in-common between the group of target images and the group of reference images and removes the joint information from the target embedding and the reference embedding. The result is a contrastive group embedding that includes a contrastive target embedding and a contrastive reference embedding with which to construct a contrastive group embedding, which is then input to a model to obtain a group caption for the target group of images.

Abstract: Systems and methods for color prediction are described. Embodiments of the present disclosure receive an image that includes an object including a color, generate a color vector based on the image using a color classification network, where the color vector includes a color value corresponding to each of a set of colors, generate a bias vector by comparing the color vector to teach of a set of center vectors, where each of the set of center vectors corresponds to a color of the set of colors, and generate an unbiased color vector based on the color vector and the bias vector, where the unbiased color vector indicates the color of the object.

Abstract: Systems and methods for intent discovery and video summarization are described. Embodiments of the present disclosure receive a video and a transcript of the video, encode the video to obtain a sequence of video encodings, encode the transcript to obtain a sequence of text encodings, apply a visual gate to the sequence of text encodings based on the sequence of video encodings to obtain gated text encodings, and generate an intent label for the transcript based on the gated text encodings.

Abstract: A group captioning system includes computing hardware, software, and/or firmware components in support of the enhanced group captioning contemplated herein. In operation, the system generates a target embedding for a group of target images, as well as a reference embedding for a group of reference images. The system identifies information in-common between the group of target images and the group of reference images and removes the joint information from the target embedding and the reference embedding. The result is a contrastive group embedding that includes a contrastive target embedding and a contrastive reference embedding with which to construct a contrastive group embedding, which is then input to a model to obtain a group caption for the target group of images.

Abstract: The present description relates to systems, methods, and non-transitory computer readable media for generating digital responses to digital queries by utilizing a classification model and query-specific analysis models. For example, the described systems can train a classification model to generate query classifications corresponding to product queries, conversational queries, and/or recommendation/purchase queries. Moreover, the described systems can apply the classification model to select pertinent models for particular queries. For example, upon classifying a product query, the described systems can utilize a neural ranking model (trained based on a set of training product specifications and training queries) to generate relevance scores for product specifications associated with a digital query.

Abstract: Systems and methods for key-phrase extraction are described. The systems and methods include receiving a transcript including a text paragraph and generating key-phrase data for the text paragraph using a key-phrase extraction network. The key-phrase extraction network is trained to identify domain-relevant key-phrase data based on domain data obtained using a domain discriminator network. The systems and methods further include generating meta-data for the transcript based on the key-phrase data.

Abstract: Techniques for training a first neural network (NN) model using a pre-trained second NN model are disclosed. In an example, training data is input to the first and second models. The training data includes masked tokens and unmasked tokens. In response, the first model generates a first prediction associated with a masked token and a second prediction associated with an unmasked token, and the second model generates a third prediction associated with the masked token and a fourth prediction associated with the unmasked token. The first model is trained, based at least in part on the first, second, third, and fourth predictions. In another example, a prediction associated with a masked token, a prediction associated with an unmasked token, and a prediction associated with whether two sentences of training data are adjacent sentences are received from each of the first and second models. The first model is trained using the predictions.

Abstract: Systems and methods for natural language processing are described. One or more aspects of a method, apparatus, and non-transitory computer readable medium include receiving a text phrase; encoding the text phrase using an encoder to obtain a hidden representation of the text phrase, wherein the encoder is trained during a first training phrase using self-supervised learning based on a first contrastive loss and during a second training phrase using supervised learning based on a second contrastive learning loss; identifying an intent of the text phrase from a predetermined set of intent labels using a classification network, wherein the classification network is jointly trained with the encoder in the second training phase; and generating a response to the text phrase based on the intent.

Abstract: A computer-implemented method is disclosed for determining one or more characteristics of a dialog between a computer system and user. The method may comprise receiving a system utterance comprising one or more tokens defining one or more words generated by the computer system; receiving a user utterance comprising one or more tokens defining one or more words uttered by a user in response to the system utterance, the system utterance and the user utterance forming a dialog context; receiving one or more utterance candidates comprising one or more tokens; for each utterance candidate, generating an input sequence combining the one or more tokens of each of the system utterance, the user utterance, and the utterance candidate; and for each utterance candidate, evaluating the generated input sequence with a model to determine a probability that the utterance candidate is relevant to the dialog context.

Abstract: The technology described herein is directed to a reinforcement learning based framework for training a natural media agent to learn a rendering policy without human supervision or labeled datasets. The reinforcement learning based framework feeds the natural media agent a training dataset to implicitly learn the rendering policy by exploring a canvas and minimizing a loss function. Once trained, the natural media agent can be applied to any reference image to generate a series (or sequence) of continuous-valued primitive graphic actions, e.g., sequence of painting strokes, that when rendered by a synthetic rendering environment on a canvas, reproduce an identical or transformed version of the reference image subject to limitations of an action space and the learned rendering policy.

Abstract: The present disclosure describes a model for large scale color prediction of objects identified in images. Embodiments of the present disclosure include an object detection network, an attention network, and a color classification network. The object detection network generates object features for an object in an image and may include a convolutional neural network (CNN), region proposal network, or a ResNet. The attention network generates an attention vector for the object based on the object features, wherein the attention network takes a query vector based on the object features, and a plurality of key vector and a plurality of value vectors corresponding to a plurality of colors as input. The color classification network generates a color attribute vector based on the attention vector, wherein the color attribute vector indicates a probability of the object including each of the plurality of colors.

Abstract: A group captioning system includes computing hardware, software, and/or firmware components in support of the enhanced group captioning contemplated herein. In operation, the system generates a target embedding for a group of target images, as well as a reference embedding for a group of reference images. The system identifies information in-common between the group of target images and the group of reference images and removes the joint information from the target embedding and the reference embedding. The result is a contrastive group embedding that includes a contrastive target embedding and a contrastive reference embedding with which to construct a contrastive group embedding, which is then input to a model to obtain a group caption for the target group of images.

Abstract: The technology described herein is directed to a reinforcement learning based framework for training a natural media agent to learn a rendering policy without human supervision or labeled datasets. The reinforcement learning based framework feeds the natural media agent a training dataset to implicitly learn the rendering policy by exploring a canvas and minimizing a loss function. Once trained, the natural media agent can be applied to any reference image to generate a series (or sequence) of continuous-valued primitive graphic actions, e.g., sequence of painting strokes, that when rendered by a synthetic rendering environment on a canvas, reproduce an identical or transformed version of the reference image subject to limitations of an action space and the learned rendering policy.

Abstract: A computer-implemented method is disclosed for determining one or more characteristics of a dialog between a computer system and user. The method may comprise receiving a system utterance comprising one or more tokens defining one or more words generated by the computer system; receiving a user utterance comprising one or more tokens defining one or more words uttered by a user in response to the system utterance, the system utterance and the user utterance forming a dialog context; receiving one or more utterance candidates comprising one or more tokens; for each utterance candidate, generating an input sequence combining the one or more tokens of each of the system utterance, the user utterance, and the utterance candidate; and for each utterance candidate, evaluating the generated input sequence with a model to determine a probability that the utterance candidate is relevant to the dialog context.

Abstract: Techniques for training a first neural network (NN) model using a pre-trained second NN model are disclosed. In an example, training data is input to the first and second models. The training data includes masked tokens and unmasked tokens. In response, the first model generates a first prediction associated with a masked token and a second prediction associated with an unmasked token, and the second model generates a third prediction associated with the masked token and a fourth prediction associated with the unmasked token. The first model is trained, based at least in part on the first, second, third, and fourth predictions. In another example, a prediction associated with a masked token, a prediction associated with an unmasked token, and a prediction associated with whether two sentences of training data are adjacent sentences are received from each of the first and second models. The first model is trained using the predictions.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for generating digital responses to digital queries by utilizing a classification model and query-specific analysis models. For example, the disclosed systems can train a classification model to generate query classifications corresponding to product queries, conversational queries, and/or recommendation/purchase queries. Moreover, the disclosed systems can apply the classification model to select pertinent models for particular queries. For example, upon classifying a product query, disclosed systems can utilize a neural ranking model (trained based on a set of training product specifications and training queries) to generate relevance scores for product specifications associated with a digital query.