Abstract: A process of a voice control method with a cloud server and a terminal device. The voice control method includes: a terminal device receiving voice information; the terminal device querying a control instruction corresponding to the voice information from a local voice database; when the control instruction corresponding to the voice information is not queried in the local voice database, the terminal device uploading the voice information to a cloud server; the cloud server parsing the control instruction corresponding to the voice information; when the control instruction corresponding to the voice information is parsed, the cloud server sending the control instruction to the terminal device; and the terminal device receiving the control instruction, and performing a corresponding operation on the basis of control instruction.

Abstract: Systems and methods for a computerized interactive voice companion include functionality that receives audio of a user's voice as the user is speaking; detects a tone and/or other relevant aspects associated with the content of the user's voice based on the audio of the user's voice as the user is speaking and determines, as the user is speaking, a response to the user speaking based on the detected tone and/or other relevant aspects associated with the content of the user's voice of the user's voice. The computerized interactive voice companion system, then orally or visually provides the response to the user automatically in real-time as a reply to the user speaking. The system may then continue the conversation based on continuing to detect the mood of the user as they speak and basing responses on this, as well as other recent user behavior detected to be relevant to the conversation.

Abstract: A faucet is provided that electronically controls the flow volume and temperature of water being dispensed. The faucet illustratively includes a faucet body and a faucet handle. In some embodiments, the faucet may include a faucet body and be voice controlled. The faucet illustratively includes an inertial motion unit sensor mounted in the faucet handle to sense spatial orientation of the faucet handle. The faucet illustratively includes an electronic flow control system to adjust flow volume and temperature of water being dispensed. The faucet illustratively includes a controller configured to receive signals from the inertial motion unit sensor and control the electronic flow control system to adjust flow volume and temperature of water being dispensed based upon the position of the faucet handle.

Abstract: Systems and processes for operating an intelligent automated assistant are provided. For example, a first speech input directed to a digital assistant is received from a user. A first response is provided based on the first speech input. A session window is initiated, wherein the session window is associated with a variable speech threshold. A second speech input is received during the session window. In accordance with a determination that the second speech input includes speech directed to the digital assistant, a duration associated with the session window is increased. In accordance with a determination that the variable speech threshold does not exceed a predetermined speech threshold, the session window is ended.

Abstract: A speech recognition system includes: a speech processor configured to identify an intention of a user included in an utterance of the user; a controller configured to identify whether a function corresponding to the intention of the utterance is performable, and if the function corresponding to the intention of the utterance is not performable, generate spoken text for requesting an other speech recognition system to perform the function corresponding to the intention of the user; and an utterance generator configured to convert the spoken text into a speech signal of an inaudible frequency band.

Abstract: A method for combining hotwords in a single utterance receives, at a first assistant-enabled device (AED), audio data corresponding to an utterance directed toward the first AED and a second AED among two or more AEDs where the audio data includes a query specifying an operation to perform. The method also detects, using a hotword detector, a first hotword assigned to the first AED that is different than a second hotword assigned to the second AED In response to detecting the first hotword, the method initiates processing on the audio data to determine that the audio data includes a term preceding the query that at least partially matches the second hotword assigned. Based on the at least partial match, the method executes a collaboration routine to cause the first AED and the second AED to collaborate with one another to fulfill the query.

Abstract: A method for determining potentially undesirable voices, in embodiments, includes: receiving audio recordings comprising voices associated with undesirable activity, and determining audio components of each of the audio recordings. The method may further comprise generating a multi-dimensional vector of the audio components for each of the plurality of audio recordings, and comparing audio components between the multi-dimensional vectors to determine clusters of multi-dimensional vectors, each cluster comprising two or more of the multi-dimensional vectors of audio components, wherein each cluster corresponds to a blacklisted voice. The method may further comprise receiving an audio recording or audio stream, and determining whether the audio recording or audio stream is associated with a voice associated with undesirable activity based on a comparison to the clusters.

Abstract: An artificial intelligence (AI) system for guiding a user interaction in a phone call or chat session. The system includes a computer running an AI algorithm, such as a machine learning algorithm, which is trained to recognize patterns in user interaction dialog which lead to satisfactory outcomes for the user. The system may operate in a completely autonomous mode, or the system connect a human agent in the loop. The algorithm adaptively guides the dialog to achieve a favorable outcome based on the current status of the dialog—including identifying a next question to ask, information to provide, or an action to take. The algorithm is trained via supervised learning using real dialog transcriptions from past user interactions which have been supplemented with decision points and outcomes. After deployment, update training may be performed on the algorithm using data captured by the system after the user interactions.

Abstract: Systems, methods, and apparatus for using a multimodal response in the dynamic generation of client device output that is tailored to a current modality of a client device is disclosed herein. Multimodal client devices can engage in a variety of interactions across the multimodal spectrum including voice only interactions, voice forward interactions, multimodal interactions, visual forward interactions, visual only interactions etc. A multimodal response can include a core message to be rendered for all interaction types as well as one or more modality dependent components to provide a user with additional information.

Abstract: Embodiments are disclosed for providing an interface with a personalized virtual personal assistant (VPA) via a computing system. The example method comprises assigning a plurality of virtual personal assistant (VPA) instances to a plurality of users, each VPA instance of the plurality of VPA instances operating concurrently. For example, assigning the plurality of VPA instances to the plurality of users may include retrieving a personalized VPA configuration for each user of the plurality of users based on a plurality of audio samples, each of the plurality of audio samples corresponding to a user of the plurality of users.

Abstract: Systems and methods of invoking functions of agents via digital assistant applications are provided. Each action-inventory can have an address template for an action by an agent. The address template can include a portion having an input variable used to execute the action. A data processing system can parse an input audio signal from a client device to identify a request and a parameter to be executed by the agent. The data processing system can select an action-inventory for the action corresponding to the request. The data processing system can generate, using the address template, an address. The address can include a substring having the parameter used to control execution of the action. The data processing system can direct an action data structure including the address to the agent to cause the agent to execute the action and to provide output for presentation.

Abstract: Disclosed are an electronic device, a system, and a controlling method thereof. The controlling method includes: receiving an input utterance, determining whether domain information and intent information are able to be extracted by analyzing the input utterance, based on at least one of the domain information and the intent information not being extracted, broadcasting a signal requesting previous utterance related information to one or more external devices connected to a same network as the electronic device, receiving the previous utterance related information from the at least one external device, extracting the domain information and the intent information based on the received previous utterance related information and the input utterance, and obtaining and outputting a response result based on the extracted domain information and intent information.

Abstract: An artificial intelligence (AI) system for guiding a user interaction in a phone call or chat session. The system includes a computer running an AI algorithm, such as a machine learning algorithm, which is trained to recognize patterns in user interaction dialog which lead to satisfactory outcomes for the user. The system may operate in a completely autonomous mode, or the system connect a human agent in the loop. The algorithm adaptively guides the dialog to achieve a favorable outcome based on the current status of the dialog—including identifying a next question to ask, information to provide, or an action to take. The algorithm is trained via supervised learning using real dialog transcriptions from past user interactions which have been supplemented with decision points and outcomes. After deployment, update training may be performed on the algorithm using data captured by the system after the user interactions.

Abstract: Methods and apparatuses are described in which unstructured computer text is analyzed for generation of customized digital documents. A server tokenizes and encodes historical user interactions and historical digital documents into multidimensional vectors. The server trains an interaction classification model using the multidimensional vectors as input to generate a classification for an input user interaction, and trains a language generation model using the multidimensional vectors as input to generate a customized digital document based upon an input user interaction. The server receives a new user interaction and encodes the new user interaction into a new multidimensional vector. The server executes the trained interaction classification model using the new vector as input to generate a digital document classification. The server executes the trained language generation model using the new vector and the classification as input to generate a customized digital document.

Abstract: A question answering system includes: a first encoder module configured to receive a question, the question including a first plurality of words, and encode the question into a first vector representation; a second encoder module configured to encode a document into a second vector representation, the document including a second plurality of words; a first reading module configured to generate a third vector representation based on the first and second vector representations; a first reformulation module configured to generate a first reformulated vector representation based on the first vector representation; a second reading module configured to generate a fifth vector representation based on the second vector representation and the first reformulated vector representation; a second reformulation module configured to generate a second reformulated vector representation based on first reformulated vector representation; and an answer module configured to determine an answer to the question based on the secon

Abstract: This disclosure proposes systems and methods for speech processing and sharing permitted entity information across speech processing systems. A first system can receive first audio data representing a first utterance. The first system can receive a first dialog identifier associated with a previous utterance. The first system can determine that the first audio data references a first entity. In some cases, the first system may not be able to resolve the first entity based on information in the first audio data. The first system can send, to a second system different from the first system, a first request for information about the first entity. The first request includes the first dialog identifier. The first system can receive first data responsive to the first request from the second system. The first system can process the first data and the first audio data to determine second data responsive to the first utterance, and output a first response representing the second data.

Abstract: System and method for providing presence of modifications in user dictation are disclosed. Exemplary implementations may: obtain primary audio information representing sound, including speech from a recording user, captured by a client computing platform; perform speech recognition on the primary audio information to generate a textual transcript; effectuate presentation of the transcript to the recording user; receive user input from the recording user; alter, based on the received user input from the recording user, a portion of the transcript to generate an altered transcript; effectuate presentation of the altered transcript in conjunction with audio playback of at least some of the primary audio information in a reviewing interface on a client computing platform; receive user input from the reviewing user; alter, based on the received user input from the reviewing user, portions of the altered transcript to generate a reviewed transcript; and store the reviewed transcript in electronic storage.

Abstract: Systems and methods can include a provider institution accounts database structured to retrievably store data corresponding to a plurality of entities including a primary entity related to a plurality of secondary entities. The data for each of the plurality of entities may include values for a plurality of fields associated with the respective entity. A data management circuit may be configured to detect that one or more fields corresponding to at least one of the secondary entities has a value which is to be updated. The data management circuit may generate a first user interface that causes the user to confirm authorization to provide the value for the one or more fields, and a second user interface including a field for receiving the value. The data management circuit may receive the value, and update the data corresponding to the at least one secondary entity in the provider institution accounts database.

Abstract: Provided are a server for providing a response message, based on a voice input of a user, and an operation method of the server. Provided are a server that recognizes health state information of a user, based on a voice input from the user, analyzes pre-stored health data, generates a response message, based on the analyzed health data, and outputs the generated response message, and an operation method of the server. Provided are a server that recognizes event information of a user from a voice input from the user, generates a response message, based on information about the type and frequency of a recognized event, and provides the generated response message, and an operation method of the server.

Abstract: Systems and methods are disclosed for managing a generative artificial intelligence (AI) model. Managing the generative AI model may include training or tuning the generative AI model before use or managing the operation of the generative AI model during use. Training or tuning a generative AI model typically requires manual review of outputs from the model based on the queries provided to the model to reduce hallucinations generated by the generative AI model. Once the model is in use, though, hallucinations still occur. Use of a confidence (whose generation is described herein) to train or tune the generative AI model and/or manage operation of the model reduces hallucinations, and thus improves performance, of the generative AI model.

Abstract: In a method for improving speech analysis between devices, a processor receives a speech input comprising audio from a speech recognition platform. A processor segments the speech input into input vectors. A processor maps the input vectors to a profile. A processor calculates affinity coefficients between each input vector and the profile. A processor aggregates the input vectors and affinity coefficients in a user profile. A processor implements the user profile in a speech recognition program.

Abstract: Systems, methods, and computer-readable media are disclosed for processing input data to determine an entity such as a product, service, user profile, etc. referenced in or otherwise relevant to a semantic context of the input data. Information related to the entity may be provided as an information package (e.g., a card) that is shareable as part of an electronic message. The card may include a representation of a network resource identifier that identifies a network resource, a network location of the network resource, and an access mechanism for accessing a representation (e.g. a product detail page) of the network resource. The network resource identifier may include one or more tags or tokens that identify an electronic messaging application provider and/or a user such as a sender or recipient of an electronic message that includes the card so as to enable compensating the provider and/or the user for a purchase of a product or service to which the card relates.

Abstract: Systems for providing network-based communication services are provided, such systems comprising a platform configured to facilitate interaction between users requesting interpretation services and individuals capable of fulfilling such requests. Both the content submitted through the system for interpretation and the resulting interpretation may be prerecorded, which facilitates the accuracy of the end-product. Methods for facilitating communication services over a network are also provided, such methods comprising the steps of receiving a request for interpretation, receiving acceptance of the request from an interpreter, receiving a response that corresponds with an interpretation of the content, and making the response available to one or more users via a server.

Abstract: Methods of authenticating a user or speaker are provided. These methods include obtaining an input speech signal and user credentials identifying the user or speaker. The input speech signal includes a single-channel signal or a multi-channel speech signal. The methods further include extracting a speech voiceprint from the input speech signal, and retrieving a reference voiceprint associated to the user credentials. The methods still further include determining a voiceprint correspondence between the speech voiceprint and the reference voiceprint, and authenticating the user or speaker depending on said voiceprint correspondence. The methods yet further include updating the reference voiceprint depending on the speech voiceprint corresponding to the authenticated user or speaker. Computer programs, systems and computing systems are also provided which are suitable for performing said methods of authenticating a user or speaker.

Abstract: The dialogue system includes a keyword acquisition unit configured to acquire an input key group containing one or a plurality of input keywords on the basis of an input of a character string, a category generation unit configured to classify the resulting sentence candidates into a plurality of categories on the basis of a comparison between the input key group acquired by the keyword acquisition unit and the storage key group contained in an FAQ database, an intra-category ranking determination unit configured to determine a priority ranking of the resulting sentence candidates within each of the categories, and a presentation unit configured to select a resulting sentence candidate of a highest priority ranking determined by the intra-category ranking determination unit from within a category of a highest priority ranking determined in advance, and present a response for prompting a user to make an additional input.

Abstract: In an approach to improve chatbot workspaces by updating chatbot workspaces through documentation updating and chatbot skill updating. Embodiments determine a chatbot knowledge base contains a set of updated information and updates a chatbot dialog decision tree based on one or more identified new topics in a set of updated information using natural language processing techniques to determine a set of intents, a set of entities, and a set of keywords. Further, embodiments identify a starting decision for traversing the chatbot dialogue decision tree based on the updated set of entities and the updated set of keywords. Additionally, embodiments interact, via a user interface, with an end user according to one or more interactions traversing the chatbot dialogue decision tree for a response.

Abstract: A method for processing multi-modal input includes receiving multiple signal inputs, each signal input having a corresponding input mode. Each signal input is processed in a series of mode-specific processing stages. Each successive mode-specific stage is associated with a successively longer scale of analysis of the signal input. A fused output is generated based on the output of a series of fused processing stages. Each successive fused processing stage is associated with a successively longer scale of analysis of the signal input. Multiple fused processing stages receive inputs from corresponding mode-specific processing stages, so that the fused output depends on the multiple of signal inputs.

Abstract: Systems and methods are described herein for alerting a user that the user will be unable to view a broadcast program based on an estimated time of arrival of the user to a media consumption device, and responsively providing the user with an option to record the broadcast program. These systems and methods are performed at least by identifying a plurality of broadcast programs that are indicated on a profile of a user, receiving an estimated time of arrival of the user to a location of a media consumption device, responsively determining whether the user will be unable to view a broadcast program of the plurality of broadcast programs, responsively providing the user with an option to record the broadcast program, and responsively causing the broadcast program to be recorded.

Abstract: A computer implemented method, system and program product is provided for linguistic alignment in specific user targeted messaging. In one embodiment, new and previously existing data about a specific user is analyzed and personality insights are determined. Location of the user is also determined. Using this location and collected data and personality insights, a multilayered set of linguistic preferences is determined for the specific user. This set is used to customize a message for the specific user based on the linguistic set and ultimately a message is sent to the specific user using a selected messaging channel.

Abstract: Devices and techniques are generally described for machine learning personalization as a service for speech processing applications. In various examples, a first request for machine learning prediction for a first speech processing skill. First skill data schema data may be received that describes content of the first speech processing skill. A first machine learning model for the first speech processing skill may be determined. A first feature definition describing a first aspect of the content may be determined. A second feature definition describing user profile data may be determined. A prediction request may be received from the first speech processing skill. First feature data may be generated according to the first feature definition and second feature data may be generated according to the second feature definition based at least in part on the prediction request.

Abstract: In some examples, an electronic device comprises an image sensor to detect a user action, an audio input device to receive an audio signal, and a processor coupled to the audio input device and the image sensor. The processor is to determine that the audio signal includes private speech based on the user action, remove the private speech from the audio signal to produce a filtered audio signal, and transmit the filtered audio signal.

Abstract: A method of twinning a primary wireless communication device with an in-vehicle wireless communication device. The method comprises inputting a plurality of phone numbers into the in-vehicle wireless communication device that comprises an eSIM storing an eSIM profile provisioned for wireless communication service in a wireless communication network, wherein each phone number is associated with a primary wireless communication device of a different user; sending a first bundle of twinning credentials by the in-vehicle wireless communication device to the wireless communication network, wherein the first bundle of twinning credentials comprises a VIN, an ICCID identifying the eSIM, an EID identifying the eSIM profile, and a first phone number of the plurality of phone numbers input into the in-vehicle communication device associated with a primary wireless communication device of a first user; and providing wireless communication service by the in-vehicle communication device based on the first phone number.

Abstract: An indication associated with an AI mode enabled by an AI system is received from a user with an affirmative opt-in status. A first task request associated with the user is identified. One or more instructions associated with the first task request are contextually recognized. The first task request is completed based on the contextually recognized one or more instructions.

Abstract: At an electronic device with a display, a microphone, and an input device: while the display is on, receiving user input via the input device, the user input meeting a predetermined condition; in accordance with receiving the user input meeting the predetermined condition, sampling audio input received via the microphone; determining whether the audio input comprises a spoken trigger; and in accordance with a determination that audio input comprises the spoken trigger, triggering a virtual assistant session.

Abstract: A voice command resolution apparatus, including a memory configured to store instructions; and a processor configured to execute the instructions to: recognize a voice command of a user in an input sound, analyze a non-speech sound included in the input sound, and determine at least one target Internet of things (IoT) device related to execution of the voice command, based on an analysis result of the non-speech sound.

Abstract: An automatic speech recognition (ASR) system that determines a textual representation of a word from a word spoken in a natural language is provided. The ASR system uses an acoustic model, a language model, and a decoder. When the ASR system receives a spoken word, the acoustic model generates word candidates for the spoken word. The language model determines an n-gram score for each word candidate. The n-gram score includes a base score and a bias score. The bias score is based on a logarithmic probability of the word candidate, where the logarithmic probability is derived using a class-based language model where the words are clustered into non-overlapping clusters according to word statistics. The decoder decodes a textual representation of the spoken word from the word candidates and the corresponding n-gram score for each word candidate.

Abstract: A conversation user interface enables patients to better understand their healthcare by integrating diagnosis, treatment, medication management, and payment, through a system that uses a virtual assistant to engage in conversation with the patient. The conversation user interface conveys a visual representation of a conversation between the virtual assistant and the patient. An identity of the patient, including preferences and medical records, is maintained throughout all interactions so that each aspect of this integrated system has access to the same information. The conversation user interface presents allows the patient to interact with the virtual assistant using natural language commands to receive information and complete task related to his or her healthcare.

Abstract: A speech-processing system receives both text data and natural-understanding data (e.g., a domain, intent, and/or entity) related to a command represented in the text data. The system uses the natural-understanding data to vary vocal characteristics in determining spectrogram data corresponding to the text data based on the natural-understanding data.

Abstract: Disclosed are systems and methods for improving interactions with and between computers in content generating, searching, hosting and/or providing systems supported by or configured with personal computing devices, servers and/or platforms. The systems interact to identify and retrieve data within or across platforms, which can be used to improve the quality of data used in processing interactions between or among processors in such systems. The disclosed systems and methods provide systems and methods for automatically allowing web browsing in a user interface of an electronic messaging system. The disclosed systems and methods automatically display electronic messages containing item information displayed in an electronic message in place of any web page links associated with the item enabling browsing of the item information from within the electronic messaging system's user interface.

Abstract: A first playback device is configured to perform functions comprising: detecting sound, identifying a wake word based on the sound as detected by the first device, receiving an indication that a second playback device has also detected the sound and identified the wake word based on the sound as detected by the second device, after receiving the indication, evaluating which of the first and second devices is to extract sound data representing the sound and thereby determining that the extraction of the sound data is to be performed by the second device over the first device, in response to the determining, foregoing extraction of the sound data, receiving VAS response data that is indicative of a given VAS response corresponding to a given voice input identified in the sound data extracted by the second device, and based on the VAS response data, output the given VAS response.

Abstract: A tool for automatically generating incident management process efficiency metrics utilizing real-time communication analysis. The tool retrieves real-time conversation data from one or more communication sources, wherein the real-time conversation data includes one or more messages having data related to an information technology (IT) incident. The tool performs conversation analysis on the one or more messages. The tool determines one or more timestamps of interest for the IT incident from the one or more messages. The tool generates one or more incident management process efficiency metrics for the IT incident utilizing the one or more timestamps of interest. The tool predicts based, at least in part, on historical conversation data, an outcome for the IT incident. The tool sends the one or more incident management process efficiency metrics and the outcome for the IT incident to a user in a notification.

Abstract: A speech interaction method includes: acquiring speech information of a user; determining a task list corresponding to the speech information, the task list comprising at least two ordered tasks; and for each task in the at least two ordered tasks, responsive to that a next task of a present task is a question-answer task, querying and sending response information of the next task to a user terminal before execution time of the next task arrives, such that the user terminal outputs the response information when the execution time of the next task arrives.

Abstract: Methods and systems for answering frequently asked questions are described. An utterance is received. A decision score that is indicative of the likelihood that the utterance is answerable according to a set of frequently asked questions and associated answers is determined for the utterance. A candidate answer from the associated answers and a selection score for the candidate answer are determined for the utterance. A total score for the candidate answer is determined based on the decision score and the selection score. The total score is indicative of the likelihood that the candidate answer is a correct answer for the utterance according to the set of frequently asked questions and associated answers.

Abstract: A system and method for the automated monitoring of inmate telephone calls as well as multi-modal search, retrieval and playback capabilities for said calls. A general term for such capabilities is multi-modal audio mining. The invention is designed to provide an efficient means for organizations such as correctional facilities to identify and monitor the contents of telephone conversations and to provide evidence of possible inappropriate conduct and/or criminal activity of inmates by analyzing monitored telephone conversations for events, including, but not limited to, the addition of third parties, the discussion of particular topics, and the mention of certain entities.

Abstract: It is an aspect of the present disclosure to provide a dialogue system capable of providing an extended function to the user by registering a new vocabulary that matches the user's preference and by changing the pre-stored conversation pattern.

Abstract: An apparatus includes a memory and processor. The memory stores previous requests and corresponding previous responses. The processor determines that a user device transmitted a new voice request, converts the voice request into a first set of text, and transmits the text to an agent device. The processor applies the machine learning algorithm to the first set of text to generate suggested responses, by identifying patterns shared by the first set of text and a subset of the previous requests that are associated with the suggested responses. The processor transmits the suggested responses to the agent device. The processor then determines that the agent device transmitted voice signals responding to the new request. The processor converts these voice signals into a second set of text. The processor stores the first set of text as a previous request, and the second set of text as a corresponding previous response.

Abstract: An omni-channel, intelligent, proactive virtual agent system and method of use are provided by which a user may engage in a conversation with the agent to interact with structured and unstructured data of an enterprise that is stored in a domain-specific world model for the enterprise.

Abstract: A method of using voice commands from a mobile device to remotely access and control a computer. The method includes receiving audio data from the mobile device at the computer. The audio data is decoded into a command. A software program that the command was provided for is determined. At least one process is executed at the computer in response to the command. Output data is generated at the computer in response to executing at least one process at the computer. The output data is transmitted to the mobile device.

Abstract: A system for assisting a customer is provided. The system comprises a database comprising multimedia files, wherein one or more tags are associated with one or more of the multimedia files. The system further comprises a virtual agent server 100 configured to receive input from a first user (customer) and a second user (customer service representative). Further, the virtual agent server 100 processes the received input to understand a context of the input. Subsequently, the virtual agent server 100 identifies one or more multimedia files based on the context and the tags associated with the multimedia files. Subsequently, the virtual agent server 100 shares at least one of the identified multimedia files with at least one of the first user and the second user to enable better understanding between the first user and the second user.

Abstract: Systems and methods for creating chatbot-enabled web forms and workflows, the method comprising, mapping web forms and workflows to intents, wherein the web forms have required fields to be completed and the workflows have required tasks to be performed; mapping the required fields and the required tasks to entities for the intents that map to the web forms and the workflows; mapping utterances to complete the required fields and perform the required tasks to the intents and the entities that map to the web forms and the workflows; and creating chatbots configured to assist users to complete the required fields and perform the required tasks using the utterances, the intents and the entities that map to the web forms and the workflows.