Abstract: A method of phrase extraction for ASR models includes obtaining audio data characterizing an utterance and a corresponding ground-truth transcription of the utterance and modifying the audio data to obfuscate a particular phrase recited in the utterance. The method also includes processing, using a trained ASR model, the modified audio data to generate a predicted transcription of the utterance, and determining whether the predicted transcription includes the particular phrase by comparing the predicted transcription of the utterance to the ground-truth transcription of the utterance. When the predicted transcription includes the particular phrase, the method includes generating an output indicating that the trained ASR model leaked the particular phrase from a training data set used to train the ASR model.

Abstract: There is a need for more effective and efficient predictive data analysis solutions and/or more effective and efficient solutions for generating an emotional sentiment score without the use of labelled data. In one example, embodiments comprise receiving an input text sequence, generating an intermediate emotional sentiment score object based at least in part on the input text sequence and by utilizing an emotional sentiment machine learning model, generating an overall emotional sentiment score based at least in part on the intermediate sentiment score object and by utilizing an emotional sentiment score transformation object, and performing one or more prediction-based actions based at least in part on the overall emotional sentiment score.

Abstract: A transformer-transducer model for unifying streaming and non-streaming speech recognition includes an audio encoder, a label encoder, and a joint network. The audio encoder receives a sequence of acoustic frames, and generates, at each of a plurality of time steps, a higher order feature representation for a corresponding acoustic frame. The label encoder receives a sequence of non-blank symbols output by a final softmax layer, and generates, at each of the plurality of time steps, a dense representation. The joint network receives the higher order feature representation and the dense representation at each of the plurality of time steps, and generates a probability distribution over possible speech recognition hypothesis. The audio encoder of the model further includes a neural network having an initial stack of transformer layers trained with zero look ahead audio context, and a final stack of transformer layers trained with a variable look ahead audio context.

Abstract: A speech module is joint trained with a knowledge module by transforming a first knowledge graph into an acoustic knowledge graph. The knowledge module is trained on the acoustic knowledge graph. Then, the knowledge module is integrated with the speech module to generate an integrated knowledge-speech module. In some instances, the speech module included in the integrated knowledge-speech module is aligned with a language module to generate an optimized speech model configured to leverage acoustic information and acoustic-based knowledge information, along with language information.

Abstract: Systems and methods are described for providing subtitles for a media content item. Subtitles are obtained, using control circuitry, for the media content item. Control circuitry determines whether a character component of the subtitles should be replaced by an image component. In response to determining that the character component of the subtitles should be replaced by an image component, control circuitry selects, from memory, an image component corresponding to the character component. Control circuitry replaces the character component of the subtitles by the image component to generate modified subtitles.

Abstract: A method for determining a category responsive to a user query is disclosed. The method includes receiving a training data set comprising a plurality of data pairs, each data pair including: (i) a query; and (ii) an associated one or more categories that are responsive to the query, wherein the one or more categories in the training data set defines a plurality of categories. The method includes training a machine learning algorithm, according to the training data set, to create a trained model, wherein training the machine learning algorithm includes: creating a first co-occurrence data structure defining co-occurrence of respective word representations of the queries with the plurality of categories, and creating a second co-occurrence data structure defining co-occurrence of respective categories in respective data pairs. The method also includes deploying the trained model to return one or more categories in response to a new query input.

Abstract: A computer-implemented method for counterfactual conversation simulation is disclosed. The computer-implemented method includes generating a system output based, at least in part, on a user input. The computer-implemented method further includes determining that a system output/user input pair is not satisfactory based, at least in part, on a system output/user input score being below a predetermined threshold. The computer-implemented method further includes generating, in response to determining the system output/user input pair is not satisfactory, a counterfactual simulation of the user input based, at least in part, on a target intent of the user input.

Abstract: Methods and systems are described herein for efficiently labeling user utterances, which may encompass any communication received from a user within a conversational interaction, and identifying novel user intents for large amounts of data. A machine learning model may be used, which is trained on embeddings of utterance data, and which may employ methods like prototypical networks and hierarchical local binary classification for hierarchical multi-label multi-class classification.

Abstract: Techniques for authenticating users at devices that interact with the users via voice input. For instance, the described techniques may allow a voice-input device to safely verify the identity of a user by engaging in a back-and-forth conversation. The device or another device coupled thereto may then verify the accuracy of the responses from the user during the conversation, as well as compare an audio signature associated with the user's responses to a pre-stored audio signature associated with the user. By utilizing multiple checks, the described techniques are able to accurately and safely authenticate the user based solely on an audible conversation between the user and the voice-input device.

Abstract: A computer-implemented method includes receiving a document insight request that requests document insights for a corpus of documents. The document insight request includes the corpus of documents, a set of entities contained within each document of the corpus of documents, and document insight request parameters that includes a confidence value threshold. The method also includes generating the document insights for the corpus of documents based on the confidence value threshold. Here, the document insights include an accuracy target and a user review rate target. The method also includes transmitting the document insights to the user device causing a graphical user interface to display the document insights on the user device.

Abstract: A speech recognition method is provided. The method may include: obtaining speech data and a speech recognition result of the speech data, the speech data including speech of a plurality of speakers, and the speech recognition result including a plurality of words; determining speaking time of each of the plurality of speakers by processing the speech data; determining, based on the speaking times of the plurality of speakers and the speech recognition result, a corresponding relationship between the plurality of words and the plurality of speakers; determining, based on the corresponding relationship, at least one conversion word from the plurality of words, each of the at least one conversion word corresponding to at least two of the plurality of speakers; and re-determining the corresponding relationship between the plurality of words and the plurality of speakers based on the at least one conversion word.

Abstract: A method of a local recognition system controlling a host device to perform one or more operations is provided. The method includes receiving, by the local recognition system, a query, performing speech recognition on the received query by implementing, by the local recognition system, a local language context comprising a set of words comprising descriptions in terms of components smaller than the words, and performing speech recognition, using the local language context, to create a transcribed query. Further, the method includes controlling the host device in dependence upon the speech recognition performed on the transcribed query.

Abstract: Disclosed is an electronic device including a communication interface, a memory, a microphone, a speaker, a display, a main processor, and a sub-processor activating the main processor by recognizing a wake-up word included in a voice input. The at least one memory stores instructions that, when executed, cause the main processor to receive a first voice input to register the wake-up word, when the first voice input does not include a specified word, to receive a second voice input including a word identical to the first voice input, through the microphone, to generate a wake-up word recognition model for recognizing the wake-up word, and to store the generated wake-up word recognition model in the at least one memory, and when the first voice input includes the specified word, to output information for requesting a third voice input, through the speaker or the display.

Abstract: A method for audio processing includes receiving a recording of a teleconference among multiple participants over a network, including an audio stream containing speech uttered by the participants and information outside the audio stream. The method further includes processing the audio stream to identify speech segments interspersed with intervals of silence, extracting speaker identifications from the information outside the audio stream in the received recording, labeling a first set of the identified speech segments from the audio stream with the speaker identifications, extracting acoustic features from the speech segments in the first set, learning a correlation between the speaker identifications labelled to the segments in the first set and the extracted acoustic features, and labeling a second set of the identified speech segments using the learned correlation, to indicate the participants who spoke during the speech segments in the second set.

Abstract: A computer-implemented method is provided for estimating output confidence of a black box Application Programming Interface (API). The method includes generating paraphrases for an input text. The method further includes calculating a distance between the input text and each respective one of the paraphrases. The method also includes sorting the paraphrases in ascending order of the distance. The method additionally includes selecting a top predetermined number of the paraphrases. The method further includes inputting the input text and the selected paraphrases into the API to obtain an output confidence score for each of the input text and the selected paraphrases. The method also includes estimating, by a hardware processor, the output confidence of the input text from a robustness of output scores of the input text and the selected paraphrases.

Abstract: Techniques for conversational-based searching are described. A system may receive a first spoken user input, and may determine that the first spoken user input corresponds to a request for information associated with an entity. The system may retrieve item results corresponding to the entity. The system may determine a suggested user input based on the retrieved item results, and may determine output data corresponding to the suggested user input. The system may send output data to a user device, where the output data includes the item results and the suggested user input. The system may receive a second spoken user input, and may determine that the second spoken user input corresponds to the suggested user input. In response, the system may send the previously determined output data to the device.

Abstract: Systems and methods are provided for generating and training a relation extraction model configured to extract document-level relations. Systems obtain a knowledge database that comprises a plurality of entity tuples and a plurality of relation types, use the knowledge database to generate annotated relation instances based on relation instances that are identified in a set of unlabeled text, generate a training dataset comprising the annotated relation instances and the set of unlabeled text, and generate the machine learning model via modular self-supervision. Systems and methods are also provided for using a relation extraction model to extract document-level relations in specific use scenarios, such as for extracting drug response relations from full-text medical research articles.

Abstract: Data such as unstructured text is received that includes a sequence of sentences. This received data is then tokenized into a plurality of tokens. The received data is segmented using a hierarchical transformer network model including a token transformer, a sentence transformer, and a segmentation classifier. The token transformer contextualizes tokens within sentences and yields sentence embeddings. The sentences transformer contextualizes sentence representations based on the sentence embedddings. The segmentation classifier predicts segments of the received data based on the contextualized sentence representations. Data can be provided which characterizes the segmentation of the received data. Related apparatus, systems, techniques and articles are also described.

Abstract: In some implementations, a language proficiency of a user of a client device is determined by one or more computers. The one or more computers then determines a text segment for output by a text-to-speech module based on the determined language proficiency of the user. After determining the text segment for output, the one or more computers generates audio data including a synthesized utterance of the text segment. The audio data including the synthesized utterance of the text segment is then provided to the client device for output.

Abstract: Automated systems and methods are provided for processing speech, comprising obtaining a trained machine learning model that has been trained using a cumulative historical data structure corresponding to at least one digitally-encoded speech representation for a plurality of telecommunications interactions conducted by a plurality of agent-side participants, which includes a first data corresponding to a score variable and a second data corresponding to a plurality of driver variables; applying the trained machine learning model: to a subset of data in the cumulative historical data structure that corresponds to a first agent-side participant of the plurality of agent-side participants, to generate a performance classification score and/or a performance direction classification score, to identify an intervention-target agent-side participant from among the plurality of agent-side participants, and to the cumulative historical data structure to identify an intervention training plan; and conducting at least on