Abstract: A method includes receiving distillation data including a plurality of out-of-domain training utterances. For each particular out-of-domain training utterance of the distillation data, the method includes generating a corresponding augmented out-of-domain training utterance, and generating, using a teacher ASR model trained on training data corresponding to a target domain, a pseudo-label corresponding to the corresponding augmented out-of-domain training utterance. The method also includes distilling a student ASR model from the teacher ASR model by training the student ASR model using the corresponding augmented out-of-domain training utterances paired with the corresponding pseudo-labels generated by the teacher ASR model.

Abstract: A computing device may receive inputted text and perform, using one or more neural networks, on-device grammar checking of a sequence of words in the inputted text, including determining, using the one or more neural networks, a grammatically correct version of the sequence of words and determining that the sequence of words does not match the grammatically correct version of the sequence of words. The computing device may, in response to determining that the sequence of words does not match the grammatically correct version of the sequence of words, output, for display at a display device, at least a portion of the grammatically correct version of the sequence of words as a suggested replacement for at least a sequence of the sequence of words in the inputted text.

Abstract: Implementations disclosed herein are directed to a hybrid federated learning (FL) technique that utilizes both federated averaging (FA) and federated distillation (FD) during a given round of FL of a given global machine learning (ML) model. Implementations may identify a population of client devices to participate in the given round of FL, determine a corresponding quantity of instances of client data available at each of the client devices that may be utilized during the given round of FL, and select different subsets of the client devices based on the corresponding quantity of instances of client data. Further, implementations may cause a first subset of the client devices to generate a corresponding FA update and a second subset of client devices to generate a corresponding FD update. Moreover, implementations may subsequently update the given global ML model based on the corresponding FA updates and the corresponding FD updates.

Abstract: A computer-implemented method includes receiving audio data that corresponds to an utterance spoken by a user and captured by a user device. The method also includes processing the audio data to determine a candidate transcription that includes a sequence of tokens for the spoken utterance. Tor each token in the sequence of tokens, the method includes determining a token embedding for corresponding token, determining a n-gram token embedding for a previous sequence of n-gram tokens, and concatenating the token embedding and the n-gram token embedding to generate a concatenated output for the corresponding token. The method also includes rescoring the candidate transcription for the spoken utterance by processing the concatenated output generated for each corresponding token in the sequence of tokens.

Abstract: The technology addresses ambiguity in neural machine translation. An encoder module receives a given text exemplar and generates an encoded representation of it. A decoder module receives the encoded representation and a set of translation prefixes. The decoder module outputs an unbounded function corresponding to a set of tokens associated with each pair of the given text exemplar and translation prefix from the set of translation prefixes. Each token is assigned a probability between 0 and 1 in a vocabulary of the exemplar at each time step. A logits module generates, based on the unbounded function, a corresponding bounded conditional probability for each token, wherein the probabilities are not normalized over the vocabulary at each time step. A loss function module having a positive loss component and a scaled negative loss component identifies whether each target text of a set of target texts is a valid translation of the exemplar.

Abstract: Systems, apparatus, and methods are provided that identify a querying user's top peers, recommends an optimal product interaction journey, and tracks the value delivered. Peers of the querying user are identified based on the common profile attributes, product interaction, and similar user initiatives. A champion peer is recommended by sorting the peers based on the cumulative quality of product interactions which is modeled by a learnable parametric equation. A metric is formulated by measuring the value delivered to the user by computing the number of product interactions aligning to the user's initiative. Most engaging product interactions (e.g., documents read, events attended) by the champion peer aligning to querying user's initiatives are identified and are recommended to the querying user as a product interaction journey.

Abstract: A joint segmenting and ASR model includes an encoder to receive a sequence of acoustic frames and generate, at each of a plurality of output steps, a higher order feature representation for a corresponding acoustic frame. The model also includes a decoder to generate based on the higher order feature representation at each of the plurality of output steps a probability distribution over possible speech recognition hypotheses, and an indication of whether the corresponding output step corresponds to an end of segment (EOS).

Abstract: Generally, the present disclosure is directed to enhanced federated learning (FL) that employs a set of clients with varying amounts of computational resources (e.g., system memory, storage, and processing bandwidth). To overcome limitations of conventional FL methods that employ a set of clients with varying amounts of computational resources, the embodiments run multi-directional knowledge distillation between the server models produced by each federated averaging (FedAvg) pool, using unlabeled server data as the distillation dataset. By co-distilling the two (or more) models frequently over the course of FedAvg rounds, information is shared between the pools without sharing model parameters. This leads to increased performance and faster convergence (in fewer federated rounds).

Abstract: A method includes receiving distillation data including a plurality of out-of-domain training utterances. For each particular out-of-domain training utterance of the distillation data, the method includes generating a corresponding augmented out-of-domain training utterance, and generating, using a teacher ASR model trained on training data corresponding to a target domain, a pseudo-label corresponding to the corresponding augmented out-of-domain training utterance. The method also includes distilling a student ASR model from the teacher ASR model by training the student ASR model using the corresponding augmented out-of-domain training utterances paired with the corresponding pseudo-labels generated by the teacher ASR model.

Abstract: Information can be distilled from a global automatic speech recognition (ASR) model to a client ASR model. Many implementations include using an RNN-T model as the ASR model, where the global ASR model includes a global encoder, a joint network, a prediction network, and where the client ASR model includes a client encoder, the joint network, and the prediction network. Various implementations include using principal component analysis (PCA) while training the global ASR model to learn a mean vector and a set of principal components corresponding to the global ASR model. Additional or alternative implementations include training the client ASR model to generate one or more predicted coefficients of the global ASR model.

Abstract: A method includes receiving a training dataset that includes one or more spoken training utterances for training an automatic speech recognition (ASR) model. Each spoken training utterance in the training dataset paired with a corresponding transcription and a corresponding target sequence of auxiliary tokens. For each spoken training utterance, the method includes generating a speech recognition hypothesis for a corresponding spoken training utterance, determining a speech recognition loss based on the speech recognition hypothesis and the corresponding transcription, generating a predicted auxiliary token for the corresponding spoken training utterance, and determining an auxiliary task loss based on the predicted auxiliary token and the corresponding target sequence of auxiliary tokens. The method also includes the ASR model jointly on the speech recognition loss and the auxiliary task loss determined for each spoken training utterance.

Abstract: A method includes receiving distillation data including a plurality of out-of-domain training utterances. For each particular out-of-domain training utterance of the distillation data, the method includes generating a corresponding augmented out-of-domain training utterance, and generating, using a teacher ASR model trained on training data corresponding to a target domain, a pseudo-label corresponding to the corresponding augmented out-of-domain training utterance. The method also includes distilling a student ASR model from the teacher ASR model by training the student ASR model using the corresponding augmented out-of-domain training utterances paired with the corresponding pseudo-labels generated by the teacher ASR model.

Abstract: Implementations disclosed herein are directed to a hybrid federated learning (FL) technique that utilizes both federated averaging (FA) and federated distillation (FD) during a given round of FL of a given global machine learning (ML) model. Implementations may identify a population of client devices to participate in the given round of FL, determine a corresponding quantity of instances of client data available at each of the client devices that may be utilized during the given round of FL, and select different subsets of the client devices based on the corresponding quantity of instances of client data. Further, implementations may cause a first subset of the client devices to generate a corresponding FA update and a second subset of client devices to generate a corresponding FD update. Moreover, implementations may subsequently update the given global ML model based on the corresponding FA updates and the corresponding FD updates.

Abstract: A computing device may receive inputted text and perform, using one or more neural networks, on-device grammar checking of a sequence of words in the inputted text, including determining, using the one or more neural networks, a grammatically correct version of the sequence of words and determining that the sequence of words does not match the grammatically correct version of the sequence of words. The computing device may, in response to determining that the sequence of words does not match the grammatically correct version of the sequence of words, output, for display at a display device, at least a portion of the grammatically correct version of the sequence of words as a suggested replacement for at least a sequence of the sequence of words in the inputted text.

Abstract: The technology addresses ambiguity in neural machine translation. An encoder module receives a given text exemplar and generates an encoded representation of it. A decoder module receives the encoded representation and a set of translation prefixes. The decoder module outputs an unbounded function corresponding to a set of tokens associated with each pair of the given text exemplar and translation prefix from the set of translation prefixes. Each token is assigned a probability between 0 and 1 in a vocabulary of the exemplar at each time step. A logits module generates, based on the unbounded function, a corresponding bounded conditional probability for each token, wherein the probabilities are not normalized over the vocabulary at each time step. A loss function module having a positive loss component and a scaled negative loss component identifies whether each target text of a set of target texts is a valid translation of the exemplar.

Abstract: The present disclosure relates generally to conformation-specific antibodies that can bind to and neutralize the activity of phosphorylated-Threonine 231-tau protein (pT231-tau). The antibodies of the present technology are useful in methods for treating a neurological disorder associated with elevated cis-pT231-tau protein expression in a subject in need thereof.

Abstract: The present disclosure relates generally to conformation-specific antibodies that can bind to and neutralize the activity of phosphorylated-Threonine 231-tau protein (pT231-tau). The antibodies of the present technology are useful in methods for treating a neurological disorder associated with elevated cis-pT231-tau protein expression in a subject in need thereof.

Abstract: A computer-implemented method includes receiving audio data that corresponds to an utterance spoken by a user and captured by a user device. The method also includes processing the audio data to determine a candidate transcription that includes a sequence of tokens for the spoken utterance. Tor each token in the sequence of tokens, the method includes determining a token embedding for corresponding token, determining a n-gram token embedding for a previous sequence of n-gram tokens, and concatenating the token embedding and the n-gram token embedding to generate a concatenated output for the corresponding token. The method also includes rescoring the candidate transcription for the spoken utterance by processing the concatenated output generated for each corresponding token in the sequence of tokens.

Abstract: The present invention encompasses IL-13 binding proteins. Specifically, the invention relates to antibodies that are chimeric, CDR grafted and humanized antibodies. Preferred antibodies have high affinity for hIL-13 and neutralize hIL-13 activity in vitro and in vivo. An antibody of the invention can be a full-length antibody or an antigen-binding portion thereof. Method of making and method of using the antibodies of the invention are also provided. The antibodies, or antibody portions, of the invention are useful for detecting hIL-13 and for inhibiting hIL-13 activity, e.g., in a human subject suffering from a disorder in which hIL-13 activity is detrimental.

Abstract: Human antibodies, preferably recombinant human antibodies, both humanized and chimeric, which specifically bind to human OX40 are disclosed. Preferred antibodies have high affinity for OX40 receptor and activate the receptor in vitro and in vivo. The antibody can be a full-length antibody or an antigen-binding portion thereof. The antibodies, or antibody portions, are useful for modulating receptor activity, e.g., in a human subject suffering from a disorder in which OX40 activity is detrimental. Nucleic acids, vectors and host cells for expressing the recombinant human antibodies are provided, and methods of synthesizing the recombinant human antibodies, are also provided.