Abstract: A system, computer program product, and a computer implemented method are provided for interfacing with a virtual dialog environment to dynamically and optimally collected context for problem diagnosis and resolution. A context model is leveraged to identify context entities, and one or more corresponding context collection mechanisms. The context model is implemented in real-time to facilitate dynamic selection of one or more of the identified context collection mechanisms, which are selectively subject to execution to resolve the problem diagnosis.

Abstract: Methods and devices for sampling applications using a touch input are described herein. In some embodiments, an electronic device detects a touch input, which may cause the electronic device to send identifiers to a backend system. The backend system may then determine an application and sample audio request associated with the received identifiers. The backend system may then receive text data representing the sample audio request and text data representing a response to the sample audio request. The backend system may generate audio data representing the received text data and send the audio data to the electronic device. If, the touch input is still occurring, the backend system may find and send more sample audio requests and the responses thereof. If the touch input stops occurring during the sample, the backend system may send instructions to the electronic device to stop outputting the sample.

Abstract: A system operable to analyze applicant features having at least one user computing device in operable communication with a user network. An application server in operable communication with the user network to host an application system for analyzing the plurality of applicant features and transmitting the plurality of applicant features to a comparator operable to compare the plurality of features to one or more other applicants.

Abstract: A speech module is joint trained with a knowledge module by transforming a first knowledge graph into an acoustic knowledge graph. The knowledge module is trained on the acoustic knowledge graph. Then, the knowledge module is integrated with the speech module to generate an integrated knowledge-speech module. In some instances, the speech module included in the integrated knowledge-speech module is aligned with a language module to generate an optimized speech model configured to leverage acoustic information and acoustic-based knowledge information, along with language information.

Abstract: Implementations can receive user input during a dialog session between a user and an automated assistant at a client device of the user and via an automated assistant platform, and in response to determining that the user input requires a user interaction with a non-assistant platform: store a state of the dialog session between the user and the automated assistant, transmit a request to initiate the user interaction to the non-assistant platform that causes an additional client device of the user to render a prompt for completing the user interaction, and receive a token associated with the user interaction from the non-assistant platform. In response to receiving the token associated with the user interaction, implementations can cause the dialog session between the user and the automated assistant to be resumed based on the stored state of the dialog session and based on the token associated with the user interaction.

Abstract: Techniques for rendering visual content, in response to one or more utterances, are described. A device receives one or more utterances that define a parameter(s) for desired output content. A system (or the device) identifies natural language data corresponding to the desired content, and uses natural language generation processes to update the natural language data based on the parameter(s). The system (or the device) then generates an image based on the updated natural language data. The system (or the device) also generates video data of an avatar. The device displays the image and the avatar, and synchronizes movements of the avatar with output of synthesized speech of the updated natural language data. The device may also display subtitles of the updated natural language data, and cause a word of the subtitles to be emphasized when synthesized speech of the word is being output.

Abstract: Methods and systems are presented for translating informal utterances into formal texts. Informal utterances may include words in abbreviation forms or typographical errors. The informal utterances may be processed by mapping each word in an utterance into a well-defined token. The mapping from the words to the tokens may be based on a context associated with the utterance derived by analyzing the utterance in a character-by-character basis. The token that is mapped for each word can be one of a vocabulary token that corresponds to a formal word in a pre-defined word corpus, an unknown token that corresponds to an unknown word, or a masked token. Formal text may then be generated based on the mapped tokens. Through the processing of informal utterances using the techniques disclosed herein, the informal utterances are both normalized and sanitized.

Abstract: Systems and methods are provided for natural language processing using neural network models and natural language virtual assistants. The system and method include receiving a natural language phrase including a word sequence, computing corresponding error probabilities that the words are errors, and for a word with a corresponding error probability above a threshold, then computing a replacement phrase with a low error probability to provide a response from the virtual assistant depending on the replacement phrase.

Abstract: Systems and methods for natural language processing are described. One or more aspects of a method, apparatus, and non-transitory computer readable medium include receiving a text phrase; encoding the text phrase using an encoder to obtain a hidden representation of the text phrase, wherein the encoder is trained during a first training phrase using self-supervised learning based on a first contrastive loss and during a second training phrase using supervised learning based on a second contrastive learning loss; identifying an intent of the text phrase from a predetermined set of intent labels using a classification network, wherein the classification network is jointly trained with the encoder in the second training phase; and generating a response to the text phrase based on the intent.

Abstract: An audio input associated with a human utterance received at the audio input device is received from a respective one of a plurality of audio input devices. Each of the plurality of audio input devices is located in a corresponding physical location within the premises. The audio input is mapped to an intent. An audible verbal response associated with the intent is provided as audio output via a selected one or more of a plurality of audio output devices. Each of the plurality of audio output devices is located in an associated physical location within the premises.

Abstract: Learning and correcting errors in text input fields to an artificial intelligence (AI) system includes receiving, by the AI system from a target system, an input text. The AI system executes a text processing operation on the input text by applying at least one transformer from a library of transformers to the input text to generate transformed text. At least one proposed correction to the input text is output by the AI system based on an analysis of the transformed text. Feedback data, associated with the at least one proposed correction, is then received from a user of the target system to iteratively learn, by the AI system, which of the transformers need be applied on future input text to accurately generate future proposed corrections on the future input text.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media for computing process flow control via determination of dialogue context between a user and an artificial intelligence assistant. Multiple defined computing process flows are received by the system. Each of the defined computing process flows describe one or more instructions to be performed. Information about the defined computing process flows and dialogue between the user and the artificial intelligence assistant are provided as a prompt to a large language model (LLM). The system receives from the LLM one or more selected commands for a defined computing process flow relating to the context of the received input from the user. The system performs the command as to the computing process flow.

Abstract: A method for creating in-block content presented in a block on a page of a workspace. The block is configured to initiate a generative process to create in-block content of a particular type. The method includes determining a selection of in-page content based on a location of the block relative to the in-page content and the particular type of in-block content. The method can include causing a generative function to create generative content of the particular type based on the selection of the in-page content. The method can further include populating a block area to present the generative content.

Abstract: In one embodiment, a method includes receiving one or more voice inputs from a first user, determining a first language register associated with the first user based on the one or more voice inputs, selecting a second language register for a voice response based on the one or more voice inputs, generating the voice response based on the second language register, and providing the voice response in response to the one or more voice inputs.

Abstract: An Online Machine Learning System (OMLS) including an Online Explanation System (OES), updated continuously, configured to provide instance level explanations and model level explanations to a user; an Online Human Expert Feedback System (OEFS), updated continuously, configured to obtain expert instance level feedback and expert model level feedback, for optimization of operation of the OMLS; an Online Machine Learning Engine (OMLE) for incorporating and utilizing one or more machine learning algorithms or models utilizing features to generate a result, and capable of incorporating and utilizing multiple different machine learning algorithms or models, wherein the OMLS is configured to perform continuous online machine learning.

Abstract: Systems and methods are described herein for a prompt engine microservice. The system may segregate a received request into a search function, a calculation function, and a schema selection function. The system may also create a new aggregation function that aggregates the results of the various processes into an input for an artificial intelligence model. By doing so, the system may process the new plurality of tasks in parallel and without the initial use of the artificial intelligence model.

Abstract: Embodiments described herein propose an approach for unsupervised structure extraction in task-oriented dialogues. Specifically, a Slot Boundary Detection (SBD) module is adopted, for which utterances from training domains are tagged with the conventional BIO schema but without the slot names. A transformer-based classifier is trained to detect the boundary of potential slot tokens in the test domain. Next, while the state number is usually unknown, it is more reasonable to assume the slot number is given when analyzing a dialogue system. The detected tokens are clustered into the number of slot of groups. Finally, the dialogue state is represented with a vector recording the modification times of every slot. The slot values are then tracked through each dialogue session in the corpus and label utterances with their dialogue states accordingly. The semantic structure is portrayed by computing the transition frequencies among the unique states.

Abstract: Exemplary embodiments relate to a system for recovering a conversation between a user and the system when the system is unable to properly respond to a user's input. The system may process the user input and determine an error condition exists. The system may query one or more storage systems to identify candidate text data based on their semantic similarity to the user input. The storage systems may store data related to past frequently entered inputs and/or user-generated inputs. Alternative text data is selected from the candidate text data, and presented to the user for confirmation.

Abstract: Embodiments described herein provide a program synthesis framework that generates code programs through a multi-turn conversation between a user and a system. Specifically, the description to solve a target problem is factorized into multiple steps, each of which includes a description in natural language (prompt) to be input into the generation model as a user utterance. The model in turn synthesizes functionally correct subprograms following the current user utterance and considering descriptions and synthesized subprograms at previous steps. The subprograms generated at the multiple steps are then combined to form an output of program in response to the target problem.

Abstract: Machine learning techniques are disclosed that allow device contact list information to be leveraged in building better models that provide more accurate assessment of user transaction risks. A computing device may receive a contact list that includes a first set of user device identifiers and generates a contact graph for that user by associating the user device identifier with the first set as first-degree contacts. The computing device may then determine that a portion of the first set of the user device identifiers are stored in a contact database (e.g. on a server) and generate a user score based on user information associated with the first set, the contact graph, and a contact graph model. The computing device may provide the user score to a transaction assessment service as in input for the transaction assessment service deciding whether particular electronic transactions are approved for processing by an electronic service provider.

Abstract: Methods and systems for rendering lists of instructions and performing actions associated with those lists are described herein. In some embodiments, an individual may request that a voice activated electronic device associated with their user account assist in performing a task using a list of instructions. The list of instructions may include metadata that indicates actions capable of being performed by additional Internet of Things (“IoT”) devices. When the instructions are rendered, an instructions speechlet may recognize the metadata and may cause one or more of the IoT devices to perform a particular action. Furthermore, the metadata may also correspond to content capable of being rendered by the voice activated electronic device to assist the individual in performing a particular step of the instructions.

Abstract: A system for proactively extracting data from complex documents is disclosed. The system may include one or more processors, an NLP device, a trained machine learning device, and a memory in communication with the one or more processors and storing instructions that, when executed by the one or more processors, are configured to cause the system to, receive one or more documents from a client device, extract one or more extractable data entries from the one or more data entries, generate, one or more normalized data entries, and proactively generate and add one or more completed data entries in place of one or more placeholders in a first document template. The system may receive a natural language prompt from a user device and determine a machine-readable semantic representation. The system may identify sensitive data entries and generate a graphical user interface identifying completed data entries and associated confidence intervals.

Abstract: A computing system and related techniques for selecting content to be automatically converted to speech and provided as an audio signal are provided. A text-to-speech request associated with a first document can be received that includes data associated with a playback position of a selector associated with a text-to-speech interface overlaid on the first document. First content associated with the first document can be determined based at least in part on the playback position, the first content including content that is displayed in the user interface at the playback position. The first document can be analyzed to identify one or more structural features associated with the first content. Speech data can be generated based on the first content and the one or more structural features.

Abstract: Techniques for providing device functionalities using device components are described. A system receives a system-generated directive from a skill system and determines a workflow to execute. The system implements a response orchestrator that operates based on the workflow that includes interception points where cross-cutting functionalities can be invoked as pluggable components. The interception points occur pre-system-generated directive, pre-device-facing directive, post-device-facing directive generation, post-device-facing directive dispatch, and the like. The system supports asynchronous execution, conditional execution, and sequential execution of components. Data determined by the cross functionality components can be used by other components for processing.

Abstract: Disclosed is a multi-turn human-machine conversation method and apparatus based on a time-sequence feature screening encoding module, belonging to the technical field of natural language processing and artificial intelligence. The technical problem to be solved by the disclosure is how to screen information for each utterance in a historical conversation so as to obtain semantic information only relevant to candidate responses and how to reserve and extract time-sequence features in the historical conversation, thus improving prediction accuracy of a multi-turn human-machine conversation system.

Abstract: A device and a method that determine an utterance type of a user utterance and generate a system response according to a determination result are achieved. A user utterance type determination unit that determines an utterance type of a user utterance, and a system response generation unit that generates a system response according to a type determination result determined by the user utterance type determination unit are included. The user utterance type determination unit determines whether the user utterance is of type A that requests all reutterances of a system utterance immediately before the user utterance, or type B that requests a reutterance of a part of the system utterance immediately before the user utterance.

Abstract: A system and method of automatically discovering unigrams in a speech data element may include receiving a language model that includes a plurality of n-grams, where each n-gram includes one or more unigrams; applying an acoustic machine-learning (ML) model on one or more speech data elements to obtain a character distribution function; applying a greedy decoder on the character distribution function, to predict an initial corpus of unigrams; filtering out one or more unigrams of the initial corpus to obtain a corpus of candidate unigrams, where the candidate unigrams are not included in the language model; analyzing the one or more first speech data elements, to extract at least one n-gram that comprises a candidate unigram; and updating the language model to include the extracted at least one n-gram.

Abstract: A method (and structure and computer product) of machine translation for processing input questions includes receiving, in a processor on a computer, an input question presented in a natural language. The input question is preprocessed to find one or more condition values for possible Structured Query Language (SQL) queries. One or more possible SQL queries are enumerated based on the one or more found condition values and a paraphrasing model is used to rank the enumerated SQL queries. The highest ranked SQL query is executed against a relational database to search for a response to the input question.

Abstract: Systems and methods for switching between natural language processing units during an interaction using a dialog delegation processor. The method includes receiving a dialog utterance from a user via a user device. The method further includes determining a general intent based on the dialog utterance using a general natural language processor. The method also includes determining at least one specific intent corresponding to the general intent using the general natural language processor. The method also includes selecting a specific natural language processor corresponding to the at least one specific intent. The method further includes determining a dialog response corresponding to the at least one of specific intent using the specific natural language processor. The method also includes generating for display the dialog response on the user device.

Abstract: Systems and methods are described to automatically generate candidate questions and responses to speed the process of response creation and editing for commercial IVAs and chatbots. Rather than create the questions and responses from scratch for a new IVA, the system uses existing questions and responses from a previous or related IVA to train a model that can generate proposed responses to provided questions. The model, or a different model, can further be trained to generate responses using data taken from company or entity-specific data sources such as websites and knowledge bases. After a set of questions and responses have been generated for an IVA they may be reviewed by one or more human reviewers to ensure they are of a suitable quality. Where no previous or related IVA exists to provide example responses, the model may be trained solely using the company or entity-specific data.

Abstract: In one embodiment, a method includes receiving a user request from a client system associated with a user, determining that the user request corresponds to a first suspended task, retrieving a first dialog state of the first suspended task from a dialog history associated with the user, generating a summary of the first suspended task based on the first dialog state using a natural-language generating (NLG) module, and sending instructions to the client system for providing the summary of the first suspended task to the user.

Abstract: A voice interaction method, where a service type set on which a user has a voice interaction intention is predicted based on a target event that can trigger voice interaction, and when a service type of a first service expressed by a voice instruction is a target service type in the service type set, the first service is executed.

Abstract: An information processing apparatus that includes a deciding unit that decides character representation related to speech-to-text conversion of a verbal action performed by a user is provided. The deciding unit decides the character representation for each detected verbal action section based on a detected non-verbal action of the user. Furthermore, an information processing method that causes a processor to execute a process for deciding character representation related to speech-to-text conversion of a verbal action performed by the user is provided. The process further includes a process for deciding the character representation for each detected verbal action section based on a detected non-verbal action performed by the user.

Abstract: Offered is a system that presents on a display screen a translation of a sentence together with an untranslated version of the sentence, and that can cause both of the displayed sentences to break apart into component parts in response to a simple user action, e.g., double-tapping on one of them. When the user selects (e.g., taps on) any portion of either version of the sentence, the system can identify a corresponding portion of the other version (in the other language). In some implementations, a user device can include both a microphone and a display screen, and an automatic speech recognition (ASR) engine can be used to transcribe the user's speech in one language (e.g., English) into text. The system can translate the resulting text into another language (e.g., Spanish) and display the translated text on the display screen along with the untranslated text. When a user selects a portion of a sentence, the system can also present information about the selected portion (e.g.

Abstract: Systems and methods for configuring a dialogue guidance graph that governs a set of operations of an automated dialogue system and that includes encoding a recall operation to a target graphical node of the graph, wherein the recall operation causes: an accessing of a temporary data storage storing a log of data of an active dialogue session between a user and the automated dialogue system, an assessment of a terminality attribute associated with the target graphical node, a determination of whether the terminality attribute is disabled or enabled, wherein if the terminality attribute is disabled, the recall operation, causes the automated dialogue system to revert the active dialogue session to the target graphical node to perform one or more dialogue-based operations between the user and the automated dialogue system that converts the terminality attribute of the target graphical node from the disabled state to the enabled state.

Abstract: A speech recognition unit converts an input utterance sequence into a confusion network sequence constituted by a k-best of candidate words of speech recognition results; a lattice generating unit generates a lattice sequence having the candidate words as internal nodes and a combination of k words among the candidate words for an identical speech as an external node, in which edges are extended between internal nodes other than internal nodes included in an identical external node, from the confusion network sequence; an integer programming problem generating unit generates an integer programming problem for selecting a path that maximizes an objective function including at least a coverage score of an important word, of paths following the internal nodes with the edges extended, in the lattice sequence; and the summary generating unit generates a high-quality summary having less speech recognition errors and low redundancy using candidate words indicated by the internal nodes included in the path selected b

Abstract: Multi-modal natural language processing systems are provided. Some systems are context-aware systems that use multi-modal data to improve the accuracy of natural language understanding as it is applied to spoken language input. Machine learning architectures are provided that jointly model spoken language input (“utterances”) and information displayed on a visual display (“on-screen information”). Such machine learning architectures can improve upon, and solve problems inherent in, existing spoken language understanding systems that operate in multi-modal contexts.

Abstract: A system for determining intent in a voice signal receives a first voice signal that indicates to perform a task. The system sends a first response that comprises a hyperlink associated with a particular webpage used to perform the task. The system receives a second voice signal that indicates whether to access the hyperlink. The system determines intent of the second voice signal by comparing keywords of the second voice signal with keywords of the first response. The system activates the hyperlink in response to determining that the keywords of the second voice signal correspond to the keywords of the first response.

Abstract: A multi-tier domain is provided for processing user voice queries and making routing decisions for generating responses, including for user voice queries that include multi-domain trigger words or phrases. When an utterance is recognized as different intents in different domains, a routing system for a domain may consider contextual signals, including those associated with other domains, to determine whether the domain is the proper one to handle the request. This determination can be performed with a statistical model specifically trained to make such determinations using the available contextual data.

Abstract: Modeling techniques to classify data sets containing personal identifiable information (PII) comprising identifiers are provided. In one technique, multiple data sets are identified, each data set containing identifiers that were generated by a computer system and that qualify as PII of a known identifier (ID) type. For each of the multiple data sets, a model is generated based on that data set and added to a set of models. A target data set that contains identifiers that were generated by the computer system and that qualify as PII of an unknown ID type is identified. A target model is generated based on the target data set. For at least one model in the set of models, a similarity operation of that model and the target model is performed. Based on the similarity operation, it is determined whether to associate the ID type of that model with the target data set.

Abstract: A system is provided for generating compliance manuals from modularized data and taxonomy-based classifications of regulatory obligations. The system comprises a plurality of databases storing regulatory compliance data and a plurality of processors that process the regulatory compliance data to generate business requirements for complying with regulatory obligations and corresponding compliance information related to the business requirements. A taxonomy engine receives business operating parameters related to a first business and identifies a subset of the business requirements and compliance information related to the business operating parameters of the first business. The taxonomy engine further aggregates the business requirements and the compliance information related to the business operation parameters and generates a compliance manual containing the business requirements and the compliance information for use by the first business.

Abstract: This document generally describes systems and methods for dynamically adapting speech recognition for individual voice queries of a user using class-based language models. The method may include receiving a voice query from a user that includes audio data corresponding to an utterance of the user, and context data associated with the user. One or more class models are then generated that collectively identify a first set of terms determined based on the context data, and a respective class to which the respective term is assigned for each respective term in the first set of terms. A language model that includes a residual unigram may then be accessed and processed for each respective class to insert a respective class symbol at each instance of the residual unigram that occurs within the language model. A transcription of the utterance of the user is then generated using the modified language model.

Abstract: Determining a dialog state of an electronic dialog that includes an automated assistant and at least one user, and performing action(s) based on the determined dialog state. The dialog state can be represented as one or more slots and, for each of the slots, one or more candidate values for the slot and a corresponding score (e.g., a probability) for each of the candidate values. Candidate values for a slot can be determined based on language processing of user utterance(s) and/or system utterance(s) during the dialog. In generating scores for candidate value(s) of a given slot at a given turn of an electronic dialog, various features are determined based on processing of the user utterance and the system utterance using a memory network. The various generated features can be processed using a scoring model to generate scores for candidate value(s) of the given slot at the given turn.

Abstract: A method to select a response in a multi-turn conversation between a user and a conversational bot. The conversation is composed of a set of events, wherein an event is a linear sequence of observations that are user speech or physical actions. Queries are processed against a set of conversations that are organized as a set of inter-related data tables, with events and observations stored in distinct tables. As the multi-turn conversation proceeds, a data model comprising an observation history, together with a hierarchy of events determined to represent the conversation up to at least one turn, is persisted. When a new input (speech or physical action) is received, it is classified using a statistical model to generate a result. The result is then mapped to an observation in the data model. Using the mapped observation, a look-up is performed into the data tables to retrieve a possible response.

Abstract: An automatic agent may be improved through feedback. A user input may be received through a user interface. A plurality of current utterance variables may be obtained by tokenizing the user input. The automatic agent may execute a machine learning policy to generate a reply to the user input based on the plurality of current utterance variables. A different reply may be obtained in response to an indication that the reply will lead to a breakdown, wherein the breakdown comprises an unhuman response from the automatic agent according to the machine learning policy. The machine learning policy may be adjusted based on the plurality of current utterance variables and the different reply.

Abstract: Systems and methods for obtaining product information via a conversational user interface. The communication channel receives communication from a user, the intent and entities of which are deduced by the NLP. These are communicated by the fulfillment API to the knowledge engine which retrieves information that fulfills the intent. The information is communicated to the fulfillment API, which converts the intent into a response, which in turn is forwarded by the NLP to the communication channel, and back to the user.

Abstract: A method, a system, and a computer program product for executing intent classification based on user feedback in a digital assistant environment. Using a natural language processor, an audio input received from user is processed. At least one implicit feedback parameter is extracted from the processed audio input. The feedback parameter classifies an intent derived from the audio input received from the user. The extracted feedback parameter is stored in a replay memory. The replay memory stores labeled data associated with the audio input received from the user. Based on the processed audio input and the labeled data, an initial response to the received audio input is determined. Modeling of the extracted implicit feedback parameter and the stored labeled data is executed. An updated response to the received audio input is generated.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining, for each of one or more categorical features, a respective vocabulary of categorical feature values of the categorical feature that should be active during processing of inputs by a machine learning model. In one aspect, a method comprises: generating a batch of output sequences, each output sequence in the batch specifying, for each of the categorical features, a respective vocabulary of categorical feature values of the categorical feature that should be active; for each output sequence in the batch, determining a performance metric of the machine learning model on a machine learning task after the machine learning model has been trained to perform the machine learning task with only the respective vocabulary of categorical feature values of each categorical feature specified by the output sequence being active.

Abstract: Systems, media, and methods for automated response to social queries comprising: monitoring queries from users, each query submitted to a vendor via an interactive chat feature of an external electronic communication platform, monitoring human responses to the queries, monitoring subsequent communications conducted via the electronic communication platform until each query is resolved; applying a first machine learning algorithm to the monitored communications to identify a query susceptible to response automation; applying a second machine learning algorithm to the query susceptible to response automation to identify one or more responses likely to resolve the query; and either i) notifying a human to respond to the query susceptible to response automation with the one or more responses likely to resolve the query, or ii) instantiating an autonomous software agent configured to respond to the query susceptible to response automation with the one or more responses likely to resolve the query.

Abstract: Techniques for implementing a chatbot that utilizes context embeddings are described. An exemplary method includes determining a next turn by: applying a language model to the utterance to determine a probability of a sequence of words, generating a context embedding for the utterance based at least on one or more of: a dialog act as defined by a chatbot definition of the chatbot, a topic vector identifying a domain of the chatbot, a previous chatbot response, and one or more slot options; performing neural language model rescoring using the determined probability of a sequence of words as a word embedding and the generated context embedding to predict an hypothesis; determining at least a name of a slot and type to be fulfilled based at least in part on the hypothesis and the chatbot definition; and determining a next turn based at least in part on the chatbot definition, any previous state, and the name of the slot and type to be fulfilled.