Abstract: Techniques for voice-based interactions are described. In an example, a device presents a user interface on a display. The device starts an operational mode of the device. The operational mode restricts voice-based interactions with the user interface to a set of commands. The set of commands is defined in a language model that is stored on the device. Further, the device receives, at a microphone of the device, audio data corresponding to a natural language utterance and generates, from the audio data, text data that corresponds to the natural language utterance. The device determines, based at least in part on the language model, that semantics of the text data correspond to a command from the set of commands and presents, on the display, an outcome of performing the command.

Abstract: Some devices may perform processing using machine learning models trained at a centralized system and distributed to the device. The centralized system may update the machine learning model and distribute the update to the device (or devices). To reduce the size of an update, the centralized system may train a model update object, which may be smaller in size than the model itself and thus more suitable for sending to the device(s). A device may receive the model update object and use it to update the on-device machine learning model; for example, by changing some parameters of the model. Parameters left unchanged during the update may retain their previous value. Thus, using the model update object to update the on-device model may result in a more accurate updated model when compared to sending an updated model compressed to a size similar to that of the model update object.

Abstract: A far-field pickup device including a device body and a microphone pickup unit is provided. The microphone pickup unit is configured to collect user speech and an echo of a first sound signal output by the device body, and transmit, to the device body, a signal obtained through digital conversion of the collected user speech and the echo. The device body includes a signal playback source, a synchronizing signal generator, a horn, a delay determining unit, and an echo cancellation unit configured to perform echo cancellation on the signal transmitted by the microphone pickup unit to obtain a collected human voice signal.

Abstract: Systems, methods, and computer-readable media are disclosed for systems and methods for language agnostic automated voice activity detection. Example methods may include determining an audio file associated with video content, generating audio segments using the audio file, the audio segments including a first segment and a second segment, and determining that the first segment includes first voice activity. Methods may include determining that the second segment comprises second voice activity, determining that voice activity is present between a first timestamp associated with the first segment and a second timestamp associated with the second segment, and generating text data representing the voice activity that is present between the first timestamp and the second timestamp.

Abstract: Embodiments of the present invention provide systems, methods, and computer storage media directed to providing talk back automation for applications installed on a mobile device. To do so actions (e.g., talk back features) can be created, via the digital assistant, by recording a series of events that are typically provided by a user of the mobile device when manually invoking the desired action. At a desired state, the user may select an object that represents the output of the application. The recording embodies the action and can be associated with a series of verbal commands that the user would typically announce to the digital assistant when an invocation of the action is desired. In response, the object is verbally communicated to the user via the digital assistant, a different digital assistant, or even another device. Alternatively, the object may be communicated to the same application or another application as input.

Abstract: A classification training system comprises a neural network configured to perform classification of input data, a training dataset including pre-segmented, labeled training samples, and a classification training module configured to train the neural network using the training dataset. The classification training module includes a forward pass processing module, and a backward pass processing module. The backward pass processing module is configured to determine whether a current frame is in a region of target (ROT), determine ROT information such as beginning and length of the ROT and update weights and biases using a cross-entropy cost function and a tunable many-or-one detection (MOOD) cost function, that comprises a tunable hyperparameter for tuning the classifier for a particular task. The backward pass module further computes a soft target value using ROT information and computes a signal output error using the soft target value and network output value.

Abstract: The systems and methods of the present disclosure generally relate to a data processing system that can identify and surface alternative requests when presented with ambiguous, unclear, or other requests to which a data processing system may not be able to respond. The data processing system can improve the efficiency of network transmissions to reduce network bandwidth usage and processor utilization by selecting alternative requests that are responsive to the intent of the original request.

Abstract: Some aspects of the invention may include a computer-implemented method for enrolling voice prints generated from audio streams, in a database. The method may include receiving an audio stream of a communication session and creating a preliminary association between the audio stream and an identity of a customer that has engaged in the communication session based on identification information. The method may further include determining a confidence level of the preliminary association based on authentication information related to the customer and if the confidence level is higher than a threshold, sending a request to compare the audio stream to a database of voice prints of known fraudsters. If the audio stream does not match any known fraudsters, sending a request to generate from the audio stream a current voice print associated with the customer and enrolling the voice print in a customer voice print database.

Abstract: A human language software defined network (SDN) control system, including: a voice to text machine learning model configured to convert user speech to text; a machine learning language processing engine configured to control the operation of a SDN controller based upon the text; and a machine learning minimal language processing engine configured to control the operation of a SDN element based upon commands from the SDN controller produced by the machine learning language processing engine.

Abstract: A personal conversationalist system includes a processor, and a computer-readable medium storing instructions which, when executed by the processor, cause the processor to operations that include receiving a first data input feed, accessing a first user profile that is as associated with a first user, detecting a conversation event when first data in the first data input feed satisfies a first conversation event condition, generating a first conversationalist persona based on the conversation event, the first user profile, and data provided in the first data input feed; and initiating a first conversation session via the first conversationalist persona by outputting a first conversationalist persona response that is based on the conversation event, the first user profile, and the data provided in the first data input feed.

Abstract: Utterance-based user interfaces can include activation trigger processing techniques for detecting activation triggers and causing execution of certain commands associated with particular command pattern activation triggers without waiting for output from a separate speech processing engine. The activation trigger processing techniques can also detect speech analysis patterns and selectively activate a speech processing engine.

Abstract: Methods and systems are provided for a natural language processing system comprising a chatbot adapted for dialog generation. In one example, the system may include a combination of a variational autoencoder (VAE) and a generative adversarial network (GAN) for generating natural responses to input queries. The VAE may convert queries into vector embeddings that may then be used by the GAN to continuously update and improve responses provided by the chatbot.

Abstract: A request including audio data is received from a voice-enabled device. A string of phonemes present in the utterance is determined through speech recognition. At a later time, a subsequent user input corresponding to the request may be received, in which the user input is associated with one or more text keywords. The subsequent user input may be obtained in response to an active request. Alternatively, feedback may not be actively elicited, but rather collected passively. However it is obtained, the one or more keywords associated with the subsequent user input may be associated with the string of phonemes to indicate that the user is saying or mean those words when they product that string of phonemes. A user-specific speech recognition key for the user account is then updated to associate the string of phonemes with these words. A general speech recognition model can also be trained using the association.

Abstract: The present disclosure relates generally to determining a language for speech recognition of a spoken utterance, received via an automated assistant interface, for interacting with an automated assistant. The system can enable multilingual interaction with the automated assistant, without necessitating a user explicitly designate a language to be utilized for each interaction. Selection of a speech recognition model for a particular language can based on one or more interaction characteristics exhibited during a dialog session between a user and an automated assistant. Such interaction characteristics can include anticipated user input types, anticipated user input durations, a duration for monitoring for a user response, and/or an actual duration of a provided user response.

Abstract: Implementations relate to determining a language for speech recognition of a spoken utterance, received via an automated assistant interface, for interacting with an automated assistant. Implementations can enable multilingual interaction with the automated assistant, without necessitating a user explicitly designate a language to be utilized for each interaction. Selection of a speech recognition model for a particular language can based on one or more interaction characteristics exhibited during a dialog session between a user and an automated assistant. Such interaction characteristics can include anticipated user input types, anticipated user input durations, a duration for monitoring for a user response, and/or an actual duration of a provided user response.

Abstract: A speech recognition method, a speech recognition apparatus, and a method and an apparatus for training a speech recognition model are provided. The speech recognition method includes: recognizing a target word speech from a hybrid speech, and obtaining, as an anchor extraction feature of a target speech, an anchor extraction feature of the target word speech based on the target word speech; obtaining a mask of the target speech according to the anchor extraction feature of the target speech; and recognizing the target speech according to the mask of the target speech.

Abstract: A system includes at least one processor to perform natural language processing on text from at least one document and assign the at least one document to at least one subjective wellbeing dimension by comparing the text from the at least one document with a subjective wellbeing dimension filter for each subjective wellbeing dimension, insert the at least one document into at least one bin, each bin associated with a particular subjective wellbeing dimension, and analyze each document in each bin associated with the particular subjective wellbeing dimension to determine a score for each subjective wellbeing dimension and an overall score that is based on each score for each subjective wellbeing dimension.

Abstract: Implementations are directed to causing a voice bot to utilize a plurality of ML layers in resolving unique personal identifier(s) for a human while the voice bot is engaged in a corresponding conversation with the human. The unique personal identifier(s) can include a unique sequence of alphanumeric characters that is personal to the human. In some implementations, ASR speech hypothes(es) corresponding to spoken utterance(s) that include the unique personal identifier(s) can be processed to generate candidate unique personal identifier(s), given alphanumeric character(s) of the candidate unique personal identifier(s) can be selected, and the voice bot can prompt the human with clarification request(s) to clarify the given alphanumeric character(s) until it is predicted to correspond to the an actual unique personal identifier(s) for the human(s). The unique personal identifier(s) can then be utilized in performance of further action(s) by the voice bot and/or other systems.

Abstract: A wake-up word for a digital assistant may be specified by a user to trigger the digital assistant to respond to the wake-up word, with the user providing one or more initial pronunciations of the wake-up word. The wake-up word may be unique, or at least not determined beforehand by a device manufacturer or developer of the digital assistant. The initial pronunciation(s) of the keyword may then be augmented with other potential pronunciations of the wake-up word that might be provided in the future, and those other potential pronunciations may then be pruned down to a threshold number of other potential pronunciations. One or more recordings of the initial pronunciation(s) of the wake-up may then be used to train a phoneme recognizer model to better recognize future instances of the wake-up word being spoken by the user or another person using the initial pronunciation or other potential pronunciations.

Abstract: Provided is a system and method for automated patient interaction. The method includes parsing a patient complaint comprising a plurality of words, determining a subset of patient queries from a plurality of patient queries based on the patient complaint and patient data, communicating the subset of patient queries to a first computing device; receiving, from the first computing device, responses to at least a portion of the subset of patient queries; generating output data based on the subset of patient queries and the responses; communicating the output data to a second computing device; receiving, from the second computing device, a user input corresponding to at least one patient query of the subset of patient queries; and training, based on the user input, at least one machine-learning algorithm configured to output at least one patient query based on at least one of the patient complaint and a subsequent patient complaint.