Abstract: In one embodiment, a method includes generating audio segments from a speech signal, generating latent representations that respectively correspond to the audio segments, the latent representations comprising a first subset and a second subset, generating quantized representations that respectively correspond to the latent representations, masking the second subset of the latent representations, using a machine-learning model to process the first subset of the latent representations and the masked second subset of the latent representations to generate contextualized representations that respectively correspond to the latent representations, pre-training the machine-learning model based on comparisons between (1) a subset of the contextualized representations that respectively correspond to the masked second subset of the latent representations and (2) a subset of the quantized representations that respectively correspond to the masked second subset of the latent representations, and training the pre-trained

Abstract: Exemplary embodiments relate to improvements to neural networks for translation and other sequence-to-sequence tasks. A convolutional neural network may include multiple blocks, each having a convolution layer and gated linear units; gating may determine what information passes through to the next block level. Residual connections, which add the input of a block back to its output, may be applied around each block. Further, an attention may be applied to determine which word is most relevant to translate next. By applying repeated passes of the attention to multiple layers of the decoder, the decoder is able to work on the entire structure of a sentence at once (with no temporal dependency). In addition to better accuracy, this configuration is better at capturing long-range dependencies, better models the hierarchical syntax structure of a sentence, and is highly parallelizable and thus faster to run on hardware.

Abstract: Examples are generally directed towards context-sensitive generation of conversational responses. Context-message-response n-tuples are extracted from at least one source of conversational data to generate a set of training context-message-response n-tuples. A response generation engine is trained on the set of training context-message-response n-tuples. The trained response generation engine automatically generates a context-sensitive response based on a user generated input message and conversational context data. A digital assistant utilizes the trained response generation engine to generate context-sensitive, natural language responses that are pertinent to user queries.

Abstract: A system for training a model to predict a sequence (e.g. a sequence of words) given a context is disclosed. A model can be trained to make these predictions using a combination of individual predictions compared to base truth and sequences of predictions based on previous predictions, where the resulting sequence is compared to the base truth sequence. In particular, the model can initially use the individual predictions to train the model. The model can then be further trained over the training data in multiple iterations, where each iteration includes two processes for each training element. In the first process, an initial part of the sequence is predicted, and the model and model parameters are updated after each prediction. In the second process, the entire remaining amount of the sequence is predicted and compared to the corresponding training sequence to adjust model parameters to encourage or discourage each prediction.

Abstract: Examples are generally directed towards context-sensitive generation of conversational responses. Context-message-response n-tuples are extracted from at least one source of conversational data to generate a set of training context-message-response n-tuples. A response generation engine is trained on the set of training context-message-response n-tuples. The trained response generation engine automatically generates a context-sensitive response based on a user generated input message and conversational context data. A digital assistant utilizes the trained response generation engine to generate context-sensitive, natural language responses that are pertinent to user queries.

Abstract: Examples are generally directed towards context-sensitive generation of conversational responses. Context-message-response n-tuples are extracted from at least one source of conversational data to generate a set of training context-message-response n-tuples. A response generation engine is trained on the set of training context-message-response n-tuples. The trained response generation engine automatically generates a context-sensitive response based on a user generated input message and conversational context data. A digital assistant utilizes the trained response generation engine to generate context-sensitive, natural language responses that are pertinent to user queries.

Abstract: Exemplary embodiments relate to improvements to neural networks for translation and other sequence-to-sequence tasks. A convolutional neural network may include multiple blocks, each having a convolution layer and gated linear units; gating may determine what information passes through to the next block level. Residual connections, which add the input of a block back to its output, may be applied around each block. Further, an attention may be applied to determine which word is most relevant to translate next. By applying repeated passes of the attention to multiple layers of the decoder, the decoder is able to work on the entire structure of a sentence at once (with no temporal dependency). In addition to better accuracy, this configuration is better at capturing long-range dependencies, better models the hierarchical syntax structure of a sentence, and is highly parallelizable and thus faster to run on hardware.

Abstract: A system for training a model to predict a sequence (e.g. a sequence of words) given a context is disclosed. A model can be trained to make these predictions using a combination of individual predictions compared to base truth and sequences of predictions based on previous predictions, where the resulting sequence is compared to the base truth sequence. In particular, the model can initially use the individual predictions to train the model. The model can then be further trained over the training data in multiple iterations, where each iteration includes two processes for each training element. In the first process, an initial part of the sequence is predicted, and the model and model parameters are updated after each prediction. In the second process, the entire remaining amount of the sequence is predicted and compared to the corresponding training sequence to adjust model parameters to encourage or discourage each prediction.

Abstract: Examples are generally directed towards automatic assessment of machine generated conversational responses. Context-message-response n-tuples are extracted from at least one source of conversational data to generate a set of multi-reference responses. A response in the set of multi-reference responses includes it context-message data pair and rating. The rating indicates a quality of the response relative to the context-message data pair. A response assessment engine generates a metric score for a machine-generated response based on an assessment metric and the set of multi-reference responses. The metric score indicates a quality of the machine-generated conversational response relative to a user-generated message and a context of the user-generated message. A response generation system of a computing device, such as a digital assistant, is optimized and adjusted based on the metric score to improve the accuracy, quality, and relevance of responses output to the user.

Abstract: Examples are generally directed towards automatic assessment of machine generated conversational responses. Context-message-response n-tuples are extracted from at least one source of conversational data to generate a set of multi-reference responses. A response in the set of multi-reference responses includes it context-message data pair and rating. The rating indicates a quality of the response relative to the context-message data pair. A response assessment engine generates a metric score for a machine-generated response based on an assessment metric and the set of multi-reference responses. The metric score indicates a quality of the machine-generated conversational response relative to a user-generated message and a context of the user-generated message. A response generation system of a computing device, such as a digital assistant, is optimized and adjusted based on the metric score to improve the accuracy, quality, and relevance of responses output to the user.

Abstract: Examples are generally directed towards context-sensitive generation of conversational responses. Context-message-response n-tuples are extracted from at least one source of conversational data to generate a set of training context-message-response n-tuples. A response generation engine is trained on the set of training context-message-response n-tuples. The trained response generation engine automatically generates a context-sensitive response based on a user generated input message and conversational context data. A digital assistant utilizes the trained response generation engine to generate context-sensitive, natural language responses that are pertinent to user queries.