Abstract: Systems and methods are described for providing subtitles for a media content item. Subtitles are obtained, using control circuitry, for the media content item. Control circuitry determines whether a character component of the subtitles should be replaced by an image component. In response to determining that the character component of the subtitles should be replaced by an image component, control circuitry selects, from memory, an image component corresponding to the character component. Control circuitry replaces the character component of the subtitles by the image component to generate modified subtitles.

Abstract: Methods and systems are described herein for efficiently labeling user utterances, which may encompass any communication received from a user within a conversational interaction, and identifying novel user intents for large amounts of data. A machine learning model may be used, which is trained on embeddings of utterance data, and which may employ methods like prototypical networks and hierarchical local binary classification for hierarchical multi-label multi-class classification.

Abstract: Techniques for authenticating users at devices that interact with the users via voice input. For instance, the described techniques may allow a voice-input device to safely verify the identity of a user by engaging in a back-and-forth conversation. The device or another device coupled thereto may then verify the accuracy of the responses from the user during the conversation, as well as compare an audio signature associated with the user's responses to a pre-stored audio signature associated with the user. By utilizing multiple checks, the described techniques are able to accurately and safely authenticate the user based solely on an audible conversation between the user and the voice-input device.

Abstract: A method for determining a category responsive to a user query is disclosed. The method includes receiving a training data set comprising a plurality of data pairs, each data pair including: (i) a query; and (ii) an associated one or more categories that are responsive to the query, wherein the one or more categories in the training data set defines a plurality of categories. The method includes training a machine learning algorithm, according to the training data set, to create a trained model, wherein training the machine learning algorithm includes: creating a first co-occurrence data structure defining co-occurrence of respective word representations of the queries with the plurality of categories, and creating a second co-occurrence data structure defining co-occurrence of respective categories in respective data pairs. The method also includes deploying the trained model to return one or more categories in response to a new query input.

Abstract: A computer-implemented method for counterfactual conversation simulation is disclosed. The computer-implemented method includes generating a system output based, at least in part, on a user input. The computer-implemented method further includes determining that a system output/user input pair is not satisfactory based, at least in part, on a system output/user input score being below a predetermined threshold. The computer-implemented method further includes generating, in response to determining the system output/user input pair is not satisfactory, a counterfactual simulation of the user input based, at least in part, on a target intent of the user input.

Abstract: A computer-implemented method includes receiving a document insight request that requests document insights for a corpus of documents. The document insight request includes the corpus of documents, a set of entities contained within each document of the corpus of documents, and document insight request parameters that includes a confidence value threshold. The method also includes generating the document insights for the corpus of documents based on the confidence value threshold. Here, the document insights include an accuracy target and a user review rate target. The method also includes transmitting the document insights to the user device causing a graphical user interface to display the document insights on the user device.

Abstract: A method of a local recognition system controlling a host device to perform one or more operations is provided. The method includes receiving, by the local recognition system, a query, performing speech recognition on the received query by implementing, by the local recognition system, a local language context comprising a set of words comprising descriptions in terms of components smaller than the words, and performing speech recognition, using the local language context, to create a transcribed query. Further, the method includes controlling the host device in dependence upon the speech recognition performed on the transcribed query.

Abstract: A speech recognition method is provided. The method may include: obtaining speech data and a speech recognition result of the speech data, the speech data including speech of a plurality of speakers, and the speech recognition result including a plurality of words; determining speaking time of each of the plurality of speakers by processing the speech data; determining, based on the speaking times of the plurality of speakers and the speech recognition result, a corresponding relationship between the plurality of words and the plurality of speakers; determining, based on the corresponding relationship, at least one conversion word from the plurality of words, each of the at least one conversion word corresponding to at least two of the plurality of speakers; and re-determining the corresponding relationship between the plurality of words and the plurality of speakers based on the at least one conversion word.

Abstract: Disclosed is an electronic device including a communication interface, a memory, a microphone, a speaker, a display, a main processor, and a sub-processor activating the main processor by recognizing a wake-up word included in a voice input. The at least one memory stores instructions that, when executed, cause the main processor to receive a first voice input to register the wake-up word, when the first voice input does not include a specified word, to receive a second voice input including a word identical to the first voice input, through the microphone, to generate a wake-up word recognition model for recognizing the wake-up word, and to store the generated wake-up word recognition model in the at least one memory, and when the first voice input includes the specified word, to output information for requesting a third voice input, through the speaker or the display.

Abstract: A method for audio processing includes receiving a recording of a teleconference among multiple participants over a network, including an audio stream containing speech uttered by the participants and information outside the audio stream. The method further includes processing the audio stream to identify speech segments interspersed with intervals of silence, extracting speaker identifications from the information outside the audio stream in the received recording, labeling a first set of the identified speech segments from the audio stream with the speaker identifications, extracting acoustic features from the speech segments in the first set, learning a correlation between the speaker identifications labelled to the segments in the first set and the extracted acoustic features, and labeling a second set of the identified speech segments using the learned correlation, to indicate the participants who spoke during the speech segments in the second set.

Abstract: A computer-implemented method is provided for estimating output confidence of a black box Application Programming Interface (API). The method includes generating paraphrases for an input text. The method further includes calculating a distance between the input text and each respective one of the paraphrases. The method also includes sorting the paraphrases in ascending order of the distance. The method additionally includes selecting a top predetermined number of the paraphrases. The method further includes inputting the input text and the selected paraphrases into the API to obtain an output confidence score for each of the input text and the selected paraphrases. The method also includes estimating, by a hardware processor, the output confidence of the input text from a robustness of output scores of the input text and the selected paraphrases.

Abstract: Techniques for conversational-based searching are described. A system may receive a first spoken user input, and may determine that the first spoken user input corresponds to a request for information associated with an entity. The system may retrieve item results corresponding to the entity. The system may determine a suggested user input based on the retrieved item results, and may determine output data corresponding to the suggested user input. The system may send output data to a user device, where the output data includes the item results and the suggested user input. The system may receive a second spoken user input, and may determine that the second spoken user input corresponds to the suggested user input. In response, the system may send the previously determined output data to the device.

Abstract: Data such as unstructured text is received that includes a sequence of sentences. This received data is then tokenized into a plurality of tokens. The received data is segmented using a hierarchical transformer network model including a token transformer, a sentence transformer, and a segmentation classifier. The token transformer contextualizes tokens within sentences and yields sentence embeddings. The sentences transformer contextualizes sentence representations based on the sentence embedddings. The segmentation classifier predicts segments of the received data based on the contextualized sentence representations. Data can be provided which characterizes the segmentation of the received data. Related apparatus, systems, techniques and articles are also described.

Abstract: Systems and methods are provided for generating and training a relation extraction model configured to extract document-level relations. Systems obtain a knowledge database that comprises a plurality of entity tuples and a plurality of relation types, use the knowledge database to generate annotated relation instances based on relation instances that are identified in a set of unlabeled text, generate a training dataset comprising the annotated relation instances and the set of unlabeled text, and generate the machine learning model via modular self-supervision. Systems and methods are also provided for using a relation extraction model to extract document-level relations in specific use scenarios, such as for extracting drug response relations from full-text medical research articles.

Abstract: In some implementations, a language proficiency of a user of a client device is determined by one or more computers. The one or more computers then determines a text segment for output by a text-to-speech module based on the determined language proficiency of the user. After determining the text segment for output, the one or more computers generates audio data including a synthesized utterance of the text segment. The audio data including the synthesized utterance of the text segment is then provided to the client device for output.

Abstract: Automated systems and methods are provided for processing speech, comprising obtaining a trained machine learning model that has been trained using a cumulative historical data structure corresponding to at least one digitally-encoded speech representation for a plurality of telecommunications interactions conducted by a plurality of agent-side participants, which includes a first data corresponding to a score variable and a second data corresponding to a plurality of driver variables; applying the trained machine learning model: to a subset of data in the cumulative historical data structure that corresponds to a first agent-side participant of the plurality of agent-side participants, to generate a performance classification score and/or a performance direction classification score, to identify an intervention-target agent-side participant from among the plurality of agent-side participants, and to the cumulative historical data structure to identify an intervention training plan; and conducting at least on

Abstract: Implementations relate to effectively localizing system responses, that include dynamic information, to target language(s), such that the system responses are grammatical and/or natural in the target language(s). Some of those implementations relate to various techniques for resource efficient generation of templates for a target language. Some versions of those implementations relate to resource efficient generation of target language natural language generation (NLG) templates and, more particularly, to techniques that enable a human user to generate a target language NLG template more efficiently and/or with greater accuracy. The more efficient target language NLG template generation enables less utilization of various client device resources and/or can mitigate the risk of flawed NLG templates being provided for live use in one or more systems.

Abstract: The present application provides a robot response method, apparatus, device and storage medium. The method includes: obtaining, by a robot, current query voice; extracting semantic information of the current query voice; matching the semantic information of the current query voice with multiple semantic information clusters stored in advance to get a matched target semantic information cluster, where each semantic information cluster includes: at least one Q&A instance, and each Q&A instance includes: semantic information corresponding to a historical query voice and a query question selected in a query list corresponding to the historical query voice; and obtaining, by the robot, the number of times each query question was selected in the target semantic information cluster, determining, according to the number of times each query question was selected, a target query question corresponding to the current query voice, and outputting a query response corresponding to the target query question.

Abstract: A method for generating synthetic speech for text through a user interface is provided. The method may include receiving one or more sentences, determining a speech style characteristic for the received one or more sentences, and outputting a synthetic speech for the one or more sentences that reflects the determined speech style characteristic. The one or more sentences and the determined speech style characteristic may be inputted to an artificial neural network text-to-speech synthesis model and the synthetic speech may be generated based on the speech data outputted from the artificial neural network text-to-speech synthesis model.

Abstract: A centralized and robust threat assessment tool is disclosed to perform comprehensive analysis of previously-stored and subsequent communication data, activity data, and other relevant information relating to inmates within a controlled environment facility. As part of the analysis, the system detects certain keywords and key interactions with the dataset in order to identify particular criminal proclivities of the inmate. Based on the identified proclivities, the system assigns threat scores to inmate that represents a relative likelihood that the inmate will carry out or be drawn to certain threats and/or criminal activities. This analysis provides a predictive tool for assessing an inmate's ability to rehabilitate. Based on the analysis, remedial measures can be taken in order to correct an inmate's trajectory within the controlled environment and increase the likelihood of successful rehabilitation, as well as to prevent potential criminal acts.