Abstract: Sentiment classification can be implemented by an entity-level multimodal sentiment classification neural network. The neural network can include left, right, and target entity subnetworks. The neural network can further include an image network that generates representation data that is combined and weighted with data output by the left, right, and target entity subnetworks to output a sentiment classification for an entity included in a network post.

Abstract: Devices and techniques are generally described for machine learning personalization as a service for speech processing applications. In various examples, a first request for machine learning prediction for a first speech processing skill. First skill data schema data may be received that describes content of the first speech processing skill. A first machine learning model for the first speech processing skill may be determined. A first feature definition describing a first aspect of the content may be determined. A second feature definition describing user profile data may be determined. A prediction request may be received from the first speech processing skill. First feature data may be generated according to the first feature definition and second feature data may be generated according to the second feature definition based at least in part on the prediction request.

Abstract: In one aspect, a playback deice is configured to identify in an audio stream, via a second wake-word engine, a false wake word for a first wake-word engine that is configured to receive as input sound data based on sound detected by a microphone. The first and second wake-word engines are configured according to different sensitivity levels for false positives of a particular wake word. Based on identifying the false wake word, the playback device is configured to (i) deactivate the first wake-word engine and (ii) cause at least one network microphone device to deactivate a wake-word engine for a particular amount of time. While the first wake-word engine is deactivated, the playback device is configured to cause at least one speaker to output audio based on the audio stream. After a predetermined amount of time has elapsed, the playback device is configured to reactivate the first wake-word engine.

Abstract: Methods, apparatus, systems, and computer-readable media are provided for storing incomplete dialog sessions between a user and an automated assistant in order that the dialog sessions can be completed in furtherance of certain actions. While interacting with an automated assistant, a user can become distracted and not complete the interaction to the point of the automated assistant performing some action. In response, the automated assistant can store the interaction as a dialog session. Subsequently, the user may express interest, directly or indirectly, in completing the dialog session, and the automated assistant can provide the user with a selectable element that, when selected, causes the dialog session to be reopened. The user can then continue the dialog session with the automated assistant in order that the originally intended action can be performed by the automated assistant.

Abstract: In non-limiting examples of the present disclosure, systems, methods and devices for providing personalized experiences to a computing device based on user input such as voice, text and gesture input are provided. Acoustic patterns associated with voice input, speech patterns, language patterns and natural language processing may be used to identify a specific user providing input from a plurality of users, identify user background characteristics and traits for the specific user, and topically categorize user input in a tiered hierarchical index. Topically categorized user input may be supplemented with user data and world knowledge and personalized responses and feedback for an identified specific user may be provided reactively and proactively.

Abstract: Techniques for improving adaptive interference cancellation (AIC) using cascaded AIC algorithms are described. To improve an accuracy of detecting speech, a device may perform a first stage of AIC to generate isolated audio data and may generate speech mask data indicating time windows when speech is detected in the isolated audio data. Based on the speech mask data, the device may perform second AIC to generate output audio data, with adaptation of the adaptive filter enabled when the speech is not detected and disabled when the speech is detected. Thus, the first AIC improves the accuracy with which the device detects that speech is present and the second AIC reduces distortion in the output audio data by not updating filter coefficient values when the speech is present. The first AIC may use playback audio data, microphone audio data or beamformed audio data as reference signals.

Abstract: Described are techniques for linking generating a skill-stored user profile, and linking same with a natural language processing (NLP) system-stored user profile. In at least some examples, a user may provide a natural language input to a NLP system. The NLP system may determine a skill is to process to perform an action responsive to the natural language input. To perform the action, the skill may require the user have a user profile stored by the skill, but the user may not have such a user profile. However, the NLP system may store a user profile for the user. The NLP system may determine the user profile stored thereby and may send, with user permission, information in the user profile to the skill. The skill may use the received information to generate and store a user profile for the user. Thereafter, the skill may provide the NLP system with a user profile identifier that the skill may use to identify the user's profile stored thereby.

Abstract: A method includes: receiving, by a computing device, a digital voice stream; receiving, by the computing device, converted text that represents the digital voice stream; identifying, by the computing device, an erroneously converted portion of the converted text; selecting, by the computing device, the erroneously converted portion for explainability processing; parsing, by the computing device, the erroneously converted portion into parts based on a predetermined parsing level; collecting, by the computing device, supplementary input data related to the erroneously converted portion; and determining, by the computing device and based on the supplemental input data, a reason why the erroneously converted portion was erroneously converted.

Abstract: Methods and apparatus for a communication system having microphones and loudspeakers to determine a noise and speech level estimate for a transformed signal, determine a SNR from the noise and speech level estimates, and determine a gain for the transformed signal to achieve a selected SNR range at a given position. In one embodiment, the gain is determined by adapting an actual gain to follow a target gain, wherein the target gain is adjusted to achieve the selected SNR range.

Abstract: Methods and systems for dynamically updating machine learning models that provide conversational responses through the use of a configuration file that defines modifications and changes to the machine learning model are disclosed. For example, the configuration file may be used to define an expected behavior and required attributes for instituting modifications and changes (e.g., via a mutation algorithm) to the machine learning model.

Abstract: A system may include first and second speech-processing systems. The first speech-processing system may process received audio data and determine that a command represented therein is associated with a second speech-processing system. The first speech-processing system may send command data to the second speech-processing system and receive response data in return. The first speech-processing system may then process the response data to determine second response data that includes an indication of the second speech-processing system and cause output of audio corresponding to the second response data.

Abstract: Present embodiments are directed to a virtual agent with improved natural language understanding (NLU) capabilities. The disclosed virtual agent enables topic selection and topic changes during natural language exchanges with a user. The virtual agent is designed to select suitable topic flows to execute based on intents identified in received user utterances, including selection of an initial topic flow in response to a topic identified in a first user utterance, as well as switching between topic flows mid-conversation based on identified topic changes. The virtual agent is also capable of considering all intents and entities conveyed during the conversation, which enables the virtual agent to avoid prompting the user to provide redundant information. Furthermore, the virtual agent is capable of executing topic flows as part of a global topic flow, which enables the virtual agent to perform a number of predefined activities as part of each interaction with the user.

Abstract: In some embodiments, a system may process a user interface to identify textual or graphical items in the interface, and may prepare a plurality of audio files containing spoken representations of the items. As the user navigates through the interface, different ones of the audio files may be selected and played, to announce text associated with items selected by the user. A computing device may periodically determine whether a cache offering the interface to users stores audio files for all of the interface's textual items, and if the cache is missing any audio files for any of the textual items, the computing device may take steps to have a corresponding audio file created.

Abstract: According to one or more aspects of the present disclosure, operations related to selecting a transcription generation technique may be disclosed. In some embodiments, the operations may include obtaining multiple user ratings that each correspond to a different one of multiple transcriptions. Each transcription may be obtained using a first transcription generation technique and may correspond to a different one of multiple communication sessions. The operations may further include selecting, for a subsequent communication session that occurs after the multiple communication sessions, a second transcription generation technique based on the user ratings. In addition, the operations may include providing the subsequent transcription to a device during the subsequent communication session.

Abstract: A processor may identify an anomaly in one or more communications. A processor may monitor the one or more communications for an utterance. A processor may perform natural language processing (NLP) on the utterance. A processor may generate an understanding of the utterance using natural language understanding (NLU). A processor may detect the anomaly from the understanding of the utterance. A processor may execute a response, responsive to detecting the anomaly.

Abstract: A computer implemented method that includes receiving a group-based audio feed for an on-going group-based communication session, the audio feed comprising a plurality of sub-feeds, each sub-feed received from a respective client user participating in the group-based communication session and comprising a plurality of sequential audio snippets; converting the audio snippets from the plurality of sub-feeds into text strings using a speech recognition engine; and causing display, to each of the respective client users while the group-based communication session is occurring, of a group-based communication session interface comprising the text strings so that the text strings are simultaneously displayed in real-time to each of the respective client users.

Abstract: Described are techniques for predicting when data associated with a user input is likely to be selected for deletion. The system may use a trained model to assist with such predictions. The trained model can be configured based on deletions associated with a user profile. An example process can including receiving user input data corresponding to the user profile, and processing the user input data to determine a user command. Based on characteristic data of the user command, the trained model can be used to determine that data corresponding to the user command is likely to be selected for deletion. The trained model can be iteratively updated based on additional user commands, including previously received user commands to delete user input data.

Abstract: Methods and apparatuses are described for intelligent dialogue recovery in virtual assistant communication sessions. A chat session is established between a virtual assistant (VA) and a remote device. A primary dialogue is created between the VA and the remote device, including (i) requesting from a first natural language processing (NLP) service an intent associated with user messages and (ii) waiting for a corresponding response. A secondary dialogue is created between the VA and the remote device that is synchronized to a state of the primary dialogue. When the response from the first NLP service succeeds, primary messages are generated and transmitted to the remote device in the primary dialogue. When the response from the first NLP service fails, an intent is requested from the second NLP service, and secondary messages are generated and transmitted to the remote device in the secondary dialogue.

Abstract: Systems and methods for efficient management of an audio/video conferences is disclosed. The method includes receiving an audio question from a first user of a plurality of users connected to a conference, recording the audio question and preventing an immediate transmission of the audio question to the plurality of users connected to the conference, analyzing the recorded question and a recorded portion of the conference to determine that the question has been answered during the recorded portion of the conference, and in response to the determining that the audio question has previously been answered, transmitting a relevant section of the recorded portion of the conference consisting of an answer to the audio question to the first user.

Abstract: A method includes applying, by at least one processor, a natural language understanding (NLU) model to an input utterance in order to obtain initial slot probability distributions. The method also includes performing, by the at least one processor, a confidence calibration by applying a calibration probability distribution to the initial slot probability distributions in order to generate calibrated slot probability distributions. The calibration probability distribution has a higher number of dimensions than the initial slot probability distributions. The method further includes identifying, by the at least one processor, uncertainties associated with words in the input utterance based on the calibrated slot probability distributions. In addition, the method includes identifying, by the at least one processor, a new concept contained in the input utterance that is not recognized by the NLU model based on the identified uncertainties.