Abstract: A voice-based digital assistant (VDA) uses a conversation intelligence (CI) manager module having a rule-based engine on conversational intelligence to process information from one or more modules to make determinations on both i) understanding the human conversational cues and ii) generating the human conversational cues, including at least understanding and generating a backchannel utterance, in a flow and exchange of human communication in order to at least one of grab or yield a conversational floor between a user and the VDA. The CI manager module uses the rule-based engine to analyze and make a determination on a conversational cue of, at least, prosody in a user's flow of speech to generate the backchannel utterance to signal any of i) an understanding, ii) a correction, iii) a confirmation, and iv) a questioning of verbal communications conveyed by the user in the flow of speech during a time frame when the user still holds the conversational floor.

Abstract: An automated interactive voice dialogue system using a human-in-the-loop design may enable human-supported interventions, where the automated system still conducts most of the interaction but enables a human agent to assume control of the dialogue and assist, if deemed necessary, so that the user may continue the interaction with little interruption or frustration. In some examples, the user of the dialogue system of this disclosure may not realize that there was a problem, and that the interaction is being or has switched from an automated dialogue system to a human. In some examples, the automated dialogue system of this disclosure may also automatically switch back to machine interaction when the human agent has resolved the situation.

Abstract: A speech recognition module receives training data of speech and creates a representation for individual words, non-words, phonemes, and any combination. A set of speech processing detectors analyze the training data of speech from humans communicating. The set of speech processing detectors detect speech parameters that are indicative of paralinguistic effects on top of enunciated words, phonemes, and non-words in the audio stream. One or more machine learning models undergo supervised machine learning on their neural network to train on how to associate one or more mark-up markers with a textual representation, for each individual word, individual non-word, individual phoneme, and any combinations of these, that was enunciated with a particular paralinguistic effect. Each mark-up marker can correspond to its own paralinguistic effect.

Abstract: Techniques for a machine learning system configured to obtain a dataset of a plurality of sample speech clips; generate a plurality of sequence; initialize a plurality of speaker embeddings and a plurality of accent embeddings; update the plurality of speaker embeddings; update the plurality of accent embeddings; generate a plurality of augmented embeddings based on the plurality of sequence embeddings, the plurality of speaker embeddings, and the plurality of accent embeddings; and generate a plurality of synthetic speech clips based on the plurality of augmented embeddings. The machine learning system may further be configured to obtain an audio waveform; decompose the audio waveform into first magnitude spectral slices and an original phase; process the first magnitude spectral slices to map the first magnitude spectral slices to second magnitude spectral slices; and generate a modified audio waveform in part by combining the second magnitude spectral slices and the original phase.

Abstract: A method, apparatus, and system for creating a script for rendering audio and/or video streams include identifying at least one prosodic speech feature in a received audio stream and/or a received language model, creating a respective prosodic speech symbol for each of the at least one identified prosodic speech features, converting the received audio stream and/or the received language model into a text stream, temporally inserting the created at least one prosodic speech symbol into the text stream, identifying in a received video stream at least one prosodic gesture of at least a portion of a body of a speaker of the received audio stream, creating at least one respective gesture symbol for each of the at least one identified prosodic gestures, and temporally inserting the created at least one gesture symbol into the text stream along with the at least one prosodic speech symbol to create a prosodic script.

Abstract: An automated interactive voice dialogue system using a human-in-the-loop design may enable human-supported interventions, where the automated system still conducts most of the interaction but enables a human agent to assume control of the dialogue and assist, if deemed necessary, so that the user may continue the interaction with little interruption or frustration. In some examples, the user of the dialogue system of this disclosure may not realize that there was a problem, and that the interaction is being or has switched from an automated dialogue system to a human. In some examples, the automated dialogue system of this disclosure may also automatically switch back to machine interaction when the human agent has resolved the situation.

Abstract: A speech recognition module receives training data of speech and creates a representation for individual words, non-words, phonemes, and any combination. A set of speech processing detectors analyze the training data of speech from humans communicating. The set of speech processing detectors detect speech parameters that are indicative of paralinguistic effects on top of enunciated words, phonemes, and non-words in the audio stream. One or more machine learning models undergo supervised machine learning on their neural network to train on how to associate one or more mark-up markers with a textual representation, for each individual word, individual non-word, individual phoneme, and any combinations of these, that was enunciated with a particular paralinguistic effect. Each mark-up marker can correspond to its own paralinguistic effect.

Abstract: A voice-based digital assistant (VDA) uses a conversation intelligence (CI) manager module having a rule-based engine on conversational intelligence to process information from one or more modules to make determinations on both i) understanding the human conversational cues and ii) generating the human conversational cues, including at least understanding and generating a backchannel utterance, in a flow and exchange of human communication in order to at least one of grab or yield a conversational floor between a user and the VDA. The CI manager module uses the rule-based engine to analyze and make a determination on a conversational cue of, at least, prosody in a user's flow of speech to generate the backchannel utterance to signal any of i) an understanding, ii) a correction, iii) a confirmation, and iv) a questioning of verbal communications conveyed by the user in the flow of speech during a time frame when the user still holds the conversational floor.

Abstract: Embodiments of the disclosed technologies include finding content of interest in an RF spectrum by automatically scanning the RF spectrum; detecting, in a range of frequencies of the RF spectrum that includes one or more undefined channels, a candidate RF segment; where the candidate RF segment includes a frequency-bound time segment of electromagnetic energy; executing a machine learning-based process to determine, for the candidate RF segment, signal characterization data indicative of one or more of: a frequency range, a modulation type, a timestamp; using the signal characterization data to determine whether audio contained in the candidate RF segment corresponds to a search criterion; in response to determining that the candidate RF segment corresponds to the search criterion, outputting, through an electronic device, data indicative of the candidate RF segment; where the data indicative of the candidate RF segment is output in a real-time time interval after the candidate RF segment is detected.

Abstract: In an embodiment, the disclosed technologies include automatically recognizing speech content of an audio stream that may contain multiple different classes of speech content, by receiving, by an audio capture device, an audio stream; outputting, by one or more classifiers, in response to an inputting to the one or more classifiers of digital data that has been extracted from the audio stream, score data; where a score of the score data indicates a likelihood that a particular time segment of the audio stream contains speech of a particular class; where the one or more classifiers use one or more machine-learned models that have been trained to recognize audio of one or more particular classes to determine the score data; using a sliding time window process, selecting particular scores from the score data; using the selected particular scores, determining and outputting one or more decisions as to whether one or more particular time segments of the audio stream contain speech of one or more particular classes

Abstract: Embodiments of the disclosed technologies include finding content of interest in an RF spectrum by automatically scanning the RF spectrum; detecting, in a range of frequencies of the RF spectrum that includes one or more undefined channels, a candidate RF segment; where the candidate RF segment includes a frequency-bound time segment of electromagnetic energy; executing a machine learning-based process to determine, for the candidate RF segment, signal characterization data indicative of one or more of: a frequency range, a modulation type, a timestamp; using the signal characterization data to determine whether audio contained in the candidate RF segment corresponds to a search criterion; in response to determining that the candidate RF segment corresponds to the search criterion, outputting, through an electronic device, data indicative of the candidate RF segment; where the data indicative of the candidate RF segment is output in a real-time time interval after the candidate RF segment is detected.

Abstract: In an embodiment, the disclosed technologies include automatically recognizing speech content of an audio stream that may contain multiple different classes of speech content, by receiving, by an audio capture device, an audio stream; outputting, by one or more classifiers, in response to an inputting to the one or more classifiers of digital data that has been extracted from the audio stream, score data; where a score of the score data indicates a likelihood that a particular time segment of the audio stream contains speech of a particular class; where the one or more classifiers use one or more machine-learned models that have been trained to recognize audio of one or more particular classes to determine the score data; using a sliding time window process, selecting particular scores from the score data; using the selected particular scores, determining and outputting one or more decisions as to whether one or more particular time segments of the audio stream contain speech of one or more particular classes

Abstract: An audio file analyzer computing system includes technologies to, among other things, localize audio events of interest (such as speakers of interest) within an audio file that includes multiple different classes (e.g., different speakers) of audio. The illustrative audio file analyzer computing system uses a seed segment to perform a semi-supervised diarization of the audio file. The seed segment is pre-selected, such as by a human person using an interactive graphical user interface.

Abstract: A method for classifying lexical stress in an utterance includes generating a feature vector representing stress characteristics of a syllable occurring in the utterance, wherein the feature vector includes a plurality of features based on prosodic information and spectral information, computing a plurality of scores, wherein each of the plurality of scores is related to a probability of a given class of lexical stress, and classifying the lexical stress of the syllable based on the plurality of scores.

Abstract: A vehicle personal assistant to engage a user in a conversational dialog about vehicle-related topics, such as those commonly found in a vehicle owner's manual, includes modules to interpret spoken natural language input, search a vehicle knowledge base and/or other data sources for pertinent information, and respond to the user's input in a conversational fashion. The dialog may be initiated by the user or more proactively by the vehicle personal assistant based on events that may be currently happening in relation to the vehicle. The vehicle personal assistant may use real-time inputs obtained from the vehicle and/or non-verbal inputs from the user to enhance its understanding of the dialog and assist the user in a variety of ways.

Abstract: The present invention relates to a method and apparatus for tailoring the output of an intelligent automated assistant. One embodiment of a method for conducting an interaction with a human user includes collecting data about the user using a multimodal set of sensors positioned in a vicinity of the user, making a set of inferences about the user in accordance with the data, and tailoring an output to be delivered to the user in accordance with the set of inferences.

Abstract: The present invention relates to a method and apparatus for tailoring the output of an intelligent automated assistant. One embodiment of a method for conducting an interaction with a human user includes collecting data about the user using a multimodal set of sensors positioned in a vicinity of the user, making a set of inferences about the user in accordance with the data, and tailoring an output to be delivered to the user in accordance with the set of inferences.

Abstract: An audio file analyzer computing system includes technologies to, among other things, localize audio events of interest (such as speakers of interest) within an audio file that includes multiple different classes (e.g., different speakers) of audio. The illustrative audio file analyzer computing system uses a seed segment to perform a semi-supervised diarization of the audio file. The seed segment is pre-selected, such as by a human person using an interactive graphical user interface.

Abstract: The present invention relates to a method and apparatus for tailoring the output of an intelligent automated assistant. One embodiment of a method for conducting an interaction with a human user includes collecting data about the user using a multimodal set of sensors positioned in a vicinity of the user, making a set of inferences about the user in accordance with the data, and tailoring an output to be delivered to the user in accordance with the set of inferences.

Abstract: The present invention relates to a method and apparatus for tailoring the output of an intelligent automated assistant. One embodiment of a method for conducting an interaction with a human user includes collecting data about the user using a multimodal set of sensors positioned in a vicinity of the user, making a set of inferences about the user in accordance with the data, and tailoring an output to be delivered to the user in accordance with the set of inferences.