Abstract: Various embodiments discussed herein enable client applications to be heavily integrated with a voice assistant in order to perform commands associated with voice utterances of users via voice assistant functionality and also seamlessly cause client applications to automatically perform native functions as part of executing the voice utterance. Such heavy integration also allows particular embodiments to support multi-modal input from a user for a single conversational interaction. In this way, client application user interface interactions, such as clicks, touch gestures, or text inputs are executed alternative or in addition to the voice utterances.

Abstract: Systems and methods are provided for an application platform with flexible permissioning.

Abstract: The present disclosure including determining that a state of a load on an occupant in a vehicle on the basis of at least one of a traveling state of the vehicle (100), an external environment state of the vehicle, and a state of the occupant in the vehicle, and executing a dialogue with the occupant by executing a dialogue program corresponding to the state of the load on the occupant.

Abstract: Implementations herein relate to information describing one or more internal states of a technical system. Implementations herein are provided for characterizing reliability of various different third party servers, at least when reporting third party device statuses, as well as adapting protocols for device ecosystems affected by such reliability. Latency can affect accuracy of device states represented by assistant devices. Certain servers can be characterized as especially delayed when reporting an updated device state in response to a user request, and, as a result, the third party server can be correlated to a metric that characterizes the relative latency of the third party server. When the metric fails to satisfy a particular threshold, a server and/or client associated with the “ecosystem” of third party devices can affirmatively operate to retrieve device state updates, rather than passively await updates from a corresponding third party server.

Abstract: Systems and methods for determining that artificial commands, in excess of a threshold value, are detected by multiple voice activated electronic devices is described herein. In some embodiments, numerous voice activated electronic devices may send audio data representing a phrase to a backend system at a substantially same time. Text data representing the phrase, and counts for instances of that text data, may be generated. If the number of counts exceeds a predefined threshold, the backend system may cause any remaining response generation functionality that particular command that is in excess of the predefined threshold to be stopped, and those devices returned to a sleep state. In some embodiments, a sound profile unique to the phrase that caused the excess of the predefined threshold may be generated such that future instances of the same phrase may be recognized prior to text data being generated, conserving the backend system's resources.

Abstract: To provide a voice analysis system capable of performing voice recognition with higher accuracy. A voice analysis system including a first voice analysis terminal and a second voice analysis terminal, the first voice analysis terminal obtaining first conversation information, and the second voice analysis terminal obtaining second conversation information, wherein the voice analysis system comprises a conversation category selection unit which compares the number of related words included in the first conversation information and the number of related words included in the second conversation information, in each conversation category, and adopts the conversation category with the larger number of related words as a correct conversation category.

Abstract: According to an embodiment, an electronic device comprises a communication module, a memory, and a processor configured to, upon obtaining a first utterance related to a memory service, prepare to store first information for the memory service for the first utterance and store the first information including essential information, sensitivity information for the first information, and an authentication method for the first information, detected from the first utterance, in the memory, and obtain a second utterance for looking up information related to the memory service, upon identifying that the obtained second utterance is one for looking up the first information, complete authentication based on the authentication method, and provide the essential information by a providing method determined based on the sensitivity information. Various other embodiments may be provided.

Abstract: The learning data generation device (10) of the present invention comprises: an end-of-talk predict unit (11) for performing: a first prediction in which it is predicted, based on utterance information on an utterance in the dialog, using the end-of-talk prediction model (16), whether the utterance is an end-of-talk utterance of the speaker; and a second prediction in which it is predicted, based on one or more prescribed rules, whether the utterance is an end-of-talk utterance; and a training data generate unit (13) for generating, when, in the first prediction it is predicted that the utterance is not an end-of-talk utterance and in the second prediction it is predicted that the utterance is an end-of-talk utterance, for the utterance information on the utterance, learning data to which training data indicating that the utterance is an end-of-talk utterance is appended.

Abstract: Systems and methods are provided for generating and operating a speech enhancement model optimized for generating noise-suppressed speech outputs for improved human listening and live captioning. A computing system obtains a speech enhancement model trained on a first training dataset to generate noise-suppressed speech outputs and an automatic speech recognition model trained on a second training dataset to generate transcription labels for spoken language utterances. A third training dataset comprising a set of spoken language utterances is applied to the speech enhancement model to obtain a first noise-suppressed speech output which is applied to the automatic speech recognition model to generate a noise-suppressed transcription output for the set of spoken language utterances.

Abstract: The present disclosure provides method and apparatus for generating a response in a human-machine conversation. A first sound input may be received in the conversation. A first audio attribute may be extracted from the first sound input, wherein the first audio attribute indicates a first condition of a user. A second sound input may be received in the conversation. A second audio attribute may be extracted from the second sound input, wherein the second audio attribute indicates a second condition of a user. A difference between the second audio attribute and the first audio attribute is determined, wherein the difference indicates a condition change of the user from the first condition to the second condition. A response to the second sound input is generated based at least on the condition change.

Abstract: Examples of the present disclosure describe systems and methods for a windowing container that enables two or more windows associated with application(s) to be grouped within the container such that the windows may behave or function uniformly as a single window. For example, responsive to a request to group two windows, a container may be generated to include the windows arranged based on one or more rules and features enabling group functions associated with the container to be performed. When a group function is performed on the container, the function may be performed to each of the windows arranged therein as if they were a single window. As new windows are grouped within or existing windows are released from the container, the container and/or windows may be rearranged based on the rules. A state of the container may be stored to enable subsequent invocation of the container after closing.

Abstract: Methods, programming, and system for modifying a slot value are described herein. In a non-limiting embodiment, an intent may be determined based on a first utterance. A first slot-value pair may be obtained for the first utterance based on the intent, the first slot-value pair including a first slot and a first value associated with the first slot. A second value associated with the first slot may be identified, the second value being identified from a second utterance that was previously received. Based on the intent and the first slot, a type of update to be performed with respect to the second value may be determined. The second value may then be updated based on the first value and the type of update.

Abstract: A system for assisting sharing of information includes circuitry to: input a plurality of sentences each representing a statement made by one of a plurality of users, the sentence being generated by speaking or writing during a meeting or by extracting from at least one of meeting data, email data, electronic file data, and chat data at any time; determine a statement type of the statement represented by each one of the plurality of sentences, the statement type being one of a plurality of statement types previously determined; select, from among the plurality of sentences being input, one or more sentences each representing a statement of a specific statement type of the plurality of types; and output a list of the selected one or more sentences as key statements of the plurality of sentences.

Abstract: Aspects of the disclosure relate to mobile device operations during an ongoing call. The mobile device operations may relate to payment processing operations. The mobile device may determine contact information associated with a party in an ongoing call. The mobile device may determine, based on the contact information, information associated with a payment transaction request. The mobile device may send, based on the determined information the payment transaction request. The mobile device may dynamically determine a user interface to be displayed during the call to facilitate the various mobile device operations.

Abstract: The present disclosure generally relates to methods and user interfaces for managing visual content at a computer system. In some embodiments, methods and user interfaces for managing visual content in media are described. In some embodiments, methods and user interfaces for managing visual indicators for visual content in media are described. In some embodiments, methods and user interfaces for inserting visual content in media are described. In some embodiments, methods and user interfaces for identifying visual content in media are described. In some embodiments, methods and user interfaces for translating visual content in media are described.

Abstract: A software-based system and method that provides a generalized scheme to voice-enable text-oriented chatbots. The system can be configured to adapt to a plurality of different types of chatbots, a plurality of different speech-to-text and text-to-speech services, a plurality of different grammars, and even a plurality of different languages. The system can further be configured to handle “HTTP complex” situations such as electronic calendars by automatically analyzing these HTTP complex situations into various sub-dialogs, which the system can then automatically use to communicate with users, and then present the final results back to the chatbot. These methods enable organizations to preserve their extensive investment in legacy chatbots while rapidly and relatively inexpensively providing voice functionality to a broader range of users.

Abstract: Systems and methods are described herein for inserting dialogue into query responses by generating and using dialogue metadata in conjunction with response templates. Metadata for each portion of dialogue of a plurality of portions of dialogue from a number of content items is stored, including information regarding the source content item, a transcript of the dialogue, and grammatical information. Upon receiving a query related to a content item, a type of response is first determined. Based on the type of response, and using the dialogue metadata, a portion of dialogue is identified for insertion into the response. The identified portion of dialogue is retrieved and inserted at an appropriate position within the response. The response is then generated for output.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for natural language processing. One of the method includes receiving a voice input from a user device; generating a recognition output; receiving a user selection of one or more terms in the recognition output; receiving a user input of one or more letters replacing the user selected one or more terms; determining suggested correction candidates based in part on the user input and the voice input; and providing one or more suggested correction candidates to the user device as suggested corrected recognition outputs.

Abstract: A computer executed process for mimicking human dialog, referred to herein as a “humanoid” or “humanoid system,” can be configured to provide automated customer support. The humanoid can identify a support issue for a customer, as well as a customer support campaign corresponding to the support issue. The humanoid can identify at least one machine learning model associated with the customer support campaign and can communicate with the customer using the at least one machine learning model. The humanoid can execute a support action to resolve the support issue.

Abstract: A media edit point selection process can include a media editing software application programmatically converting speech to text and storing a timestamp-to-text map. The map correlates text corresponding to speech extracted from an audio track for the media clip to timestamps for the media clip. The timestamps correspond to words and some gaps in the speech from the audio track. The probability of identified gaps corresponding to a grammatical pause by the speaker is determined using the timestamp-to-text map and a semantic model. Potential edit points corresponding to grammatical pauses in the speech are stored for display or for additional use by the media editing software application. Text can optionally be displayed to a user during media editing.

Abstract: There is much data that is currently not being captured during user interaction with mobile apps that could provide insight into how to effectively address a user concern. Capturing such data may allow auto-adjustments of operational responses provided by mobile apps in response to detecting anomalous user inputs. Such anomalous user inputs may include keyboard dynamics or mobile device movement that deviate from an average or user specific levels. Such anomalous user inputs may indicate that a user concern is particularly urgent. Auto-adjustments to operation of a mobile app may include initiating targeted chatbot or live chat responses.

Abstract: Techniques for decreasing (or eliminating) the possibility of a skill performing an action that is not responsive to a corresponding user input are described. A system may train one or more machine learning models with respect to user inputs, which resulted in incorrect actions being performed by skills, and corresponding user inputs, which resulted in the correct action being performed. The system may use the trained machine learning model(s) to rewrite user inputs that, if not rewritten, may result in incorrect actions being performed. The system may implement the trained machine learning model(s) with respect to ASR output text data to determine if the ASR output text data corresponds (or substantially corresponds) to previous ASR output text data that resulted in an incorrect action being performed.

Abstract: Methods, apparatus, systems, and computer-readable media for engaging an automated assistant to perform multiple tasks through a multitask command. The multitask command can be a command that, when provided by a user, causes the automated assistant to invoke multiple different agent modules for performing tasks to complete the multitask command. During execution of the multitask command, a user can provide input that can be used by one or more agent modules to perform their respective tasks. Furthermore, feedback from one or more agent modules can be used by the automated assistant to dynamically alter tasks in order to more effectively use resources available during completion of the multitask command.

Abstract: An electronic device is provided. The electronic device includes a display, a communication circuit, a processor operatively connected to the display and the communication circuit, and a memory operatively connected to the processor. The memory stores instructions that, when executed, cause the processor to receive information about a time interval and user interface information, which are associated with a response to a user utterance input to a first external electronic device, from a second external electronic device through the communication circuit, to determine whether the display is in an active state within the time interval, and to provide a first user interface corresponding to the user interface information through the display based on the determination that the display is in the active state within the time interval.

Abstract: One or more computing devices, systems, and/or methods for user identification and authorization are provided. In an example, a voice command to perform an action is detected. A voice profile associated with a user is identified based upon voice characteristics of the voice command. In response to determining that the voice profile is not linked to an account associated with the action, the user is prompted for an identifier associated with a device for creating the account through the device. In response to receiving the identifier from the user, the identifier is utilized to facilitate creation of the account through the device.

Abstract: Systems and methods for correcting a voice query based on a subsequent voice query with a lower pronunciation rate. In some aspects, the systems and methods calculate first and second pronunciation rates of first and second voice queries. The systems and methods determine that the second pronunciation rate is lower than the first pronunciation rate and determine a first candidate pronunciation time for a first candidate word from the first voice query. The systems and methods determine a second candidate pronunciation time, adjusted to the first pronunciation rate, for the second candidate word from the second voice query. The systems and methods determine that the first candidate pronunciation time matches the second candidate pronunciation time and generate a third voice query based on the first voice query by replacing the first candidate word with the second candidate word.

Abstract: In a method for efficiently and accurately measuring the intelligibility of speech, a user may utter a sample text, and an automatic speech assessment (ASA) system may receive an acoustic signal encoding the utterance. An automatic speech recognition (ASR) module may generate an N-best output corresponding to the utterance and generate an intelligibility score representing the intelligibility of the utterance based on the N-best output and the sample text. Generating the intelligibility score may involve (1) calculating conditional intelligibility value(s) for the N recognition result(s), and (2) determining the intelligibility score based on the conditional intelligibility value of the most intelligible recognition result. Optionally, the process of generating the intelligibility score may involve adjusting the intelligibility score to account for environmental information (e.g., a pronunciation score for the user's speech and/or a confidence score assigned to the 1-best recognition result).

Abstract: A device may receive audio data based on a capturing of sounds associated with a structure. The device may obtain a model associated with the structure. The model may have been trained to receive the audio data as input, determine a score that identifies a likelihood that a sound is present in the audio data, and identify the sound based on the score. The device may determine at least one parameter associated with the sound. The device may generate a metric based on the at least one parameter associated with the sound, and perform an action based on the metric.

Abstract: A method is for creating a controller for controlling an output unit from information in the context of medical diagnostics and therapy. The method includes providing a learning processing apparatus designed via an algorithm to recognize spoken words; providing on, or in, the learning processing apparatus, an untrained controller, designed to be trained via machine learning; providing a number of speech recordings, each including a communication during a medical procedure, wherein the speech recordings concern comparable medical procedures; performing a speech analysis of the speech recordings; and training the untrained controller according to a machine learning principle based upon the speech analysis of the speech recordings.

Abstract: A computer implemented method for the automated analysis or use of data is implemented by a voice assistant. The method comprises the steps of: (a) storing in a memory a structured, machine-readable representation of data that conforms to a machine-readable language (‘machine representation’); the machine representation including representations of user speech or text input to a human/machine interface; and (b) automatically processing the machine representations to analyse the user speech or text input.

Abstract: Systems and methods of generating voice-based software applications are provided. A system can receive, from an application developer computing device, a request to build a voice-based software application. The system can select an application template from a plurality of application templates. The selected application template can include a module that corresponds to a function of the voice-based software application. The system can provide the selected application template to the application developer computing device. The system can receive, from the application developer computing device, an input for a field of the at least one module of the selected application template. The system can generate the voice-based software application based on the selected application template and the input for the at least one field of the at least one module of the selected application template.

Abstract: A voice-operated system includes a processing device, and a controller that communicates with the processing device. The processing device includes a first processor to perform: displaying, on an operating panel, an operation screen for instructing a process for execution by the processing device, to receive user's instruction; and executing a process corresponding to a command received from the controller. The controller includes a second processor to perform: generating the command for the processing device based on an input voice; and transmitting the command to the processing device. The generation of the command includes, when the voice instructs the processing device to execute a first process, generating a first command for instructing the operating panel to display the operation screen for instructing execution of the first process; and when the voice instructs the processing device to execute a second process, generating a second command for instructing execution of the second process.

Abstract: Implementations can receive audio data corresponding to a spoken utterance of a user, process the audio data to generate a plurality of speech hypotheses, determine an action to be performed by an automated assistant based on the speech hypotheses, and cause the computing device to render an indication of the action. In response to the computing device rendering the indication, implementations can receive additional audio data corresponding to an additional spoken utterance of the user, process the additional audio data to determine that a portion of the spoken utterance is similar to an additional portion of the additional spoken utterance, supplant the action with an alternate action, and cause the automated assistant to initiate performance of the alternate action. Some implementations can determine whether to render the indication of the action based on a confidence level associated with the action.

Abstract: Methods, apparatus, systems and articles of manufacture for recognizing speech are disclosed. An example system includes one or more processors to execute instructions to: identify a plurality of phonemes in a speech signal; perform a comparison of a subset of the phonemes to a phonetic string, the phonetic string representative of at least a portion of a wake up phrase; determine if one or more of the phonemes of the subset correspond to the wake up phrase based on the comparison; and generate a hypothesis of a command included in the speech signal by excluding the wake up phrase when one or more of the phonemes of the subset correspond to the wake up phrase or a portion of the wake up phrase.

Abstract: A computer system may connect to various customer-facing devices and manage or automate the order process between a retail store and the customer. The computer system may perform the dialogue and receive an order for items from the retail store and may perform quality control monitoring of the dialogue between customers and employees taking orders. The ordering system may utilize the ordered items in combination with various contextual cues to determine a customer identity which may then be linked to past orders and/or various order preferences. Based on the determined customer identity, the system may provide recommendations of additional order items or order alterations to the customer before personally identifying information has been collected from the customer. The determination of the customer identity and the determination of recommendations may be performed by machine learning algorithms that were trained on customer data and the retail store products.

Abstract: The present invention relates to a drive for an electric application such as an electric motor, said drive including at least one microphone for registering noise signals occurring at the drive, wherein the microphone is connectable to a computing device for analysing the registered noise signals. The registered noise signals may be used for a maintenance process of the drive and/or a fine-tuning process of a drive control method of the drive. The present invention also relates to a maintenance process, in particular a predictive maintenance process for a corresponding drive. Furthermore, the present invention relates to a process for fine tuning a drive control method of a corresponding drive.

Abstract: A device and a method of providing a to-do list of a user are provided. The device includes a controller configured to collect behavior information about behavior between the user and another user, the behavior being performed by using the device, generate a to-do list of the user based on the collected behavior information, and determine an unperformed task not performed by the user from among at least one task in the to-do list by using log information about an operation of the device, and an output unit configured to output notification information in a dialogue style, along with a notification reason for notifying the determined unperformed task.

Abstract: An example emergency vehicle siren system can include: one or more emergency lights; a siren; and a controller including a processor and memory, the memory encoding instructions which, when executed by the processor, cause the controller to modify a sound of the siren based upon a context of the vehicle, the context including at least one of a speed and location of the vehicle.

Abstract: A method for transcription is performed by a computer. The method includes: accepting input of a voice after causing a display unit to display a sentence including a plurality of words; acquiring first sound information being information concerning sounds corresponding to the sentence; acquiring second sound information being information concerning sounds of the voice accepted in the accepting; specifying a portion in the first sound information having a prescribed similarity to the second sound information; and correcting a character string in the sentence corresponding to the specified portion based on a character string corresponding to the second sound information.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for natural language processing. One of the methods includes receiving a first voice input from a user device; generating a first recognition output; receiving a user selection of one or more terms in the first recognition output; receiving a second voice input spelling a correction of the user selection; determining a corrected recognition output for the selected portion; and providing a second recognition output that merges the first recognition output and the corrected recognition output.

Abstract: Configurable core domains of a speech processing system are described. A core domain output data format for a given command is originally configured with default content portions. When a user indicates additional content should be output for the command, the speech processing system creates a new output data format for the core domain. The new output data format is user specific and includes both default content portions as well as user preferred content portions.

Abstract: A natural language interfacing system may use a knowledge acquisition tool to obtain structured representations from user input text. The system may initiate interaction with a request for input and a partial statement with blank text slots labeled by field types. The system may receive input text to fill in a slot of the partial statement and perform semantic parsing on the input text to identify a trigger concept. The system may generate a list of templates defining different semantic frames for the trigger concept. A generated template may include additional generated slots and/or suggested slot-fillers to guide user input. In response to a template selection, the partial statement includes the trigger concept annotated with a semantic frame. This process is repeated by iteratively updating the list of templates until the statement is completed. The statement is mapped to a structured representation including semantic frames.

Abstract: Certain aspects of the disclosure are directed to context aggregation in a data communications network. According to a specific example, process user-data communications between a client station and another station participating in data communications via the data communications services can be processed, where the client station is associated with one of a plurality of client entities configured and arranged to interface with a data communications server providing data communications services. Context information can be aggregated for each respective user-data communication between the client station and the participating station, where the context information corresponds to at least one communications-specific characteristic associated with the user-data communications.

Abstract: Disclosed is a voice recognition device and method. According to the disclosure, the voice recognition device, upon failing to grasp the intent of the user's utterance from the original utterance which is divided into a head utterance and a tail utterance, figures out the intent from the head utterance to thereby complete the original utterance and provides the result of voice recognition processing on the original utterance. According to an embodiment, the voice recognition device may be related to artificial intelligence (AI) modules, robots, augmented reality (AR) devices, virtual reality (VR) devices, and 5G service-related devices.

Abstract: Techniques for detecting certain acoustic events from audio data are described. A system may perform event aggregation for certain types of events before sending an output to a device representing the event is detected. The system may bypass the event aggregation process for certain types of events that the system may detect with a high level of confidence. In such cases, the system may send an output to the device when the event is detected. The system may be used to detect acoustic events representing presence of a person or other harmful circumstances (such as, fire, smoke, etc.) in a home, an office, a store, or other types of indoor settings.

Abstract: Described herein is a system for automatically detecting and assigning action items in a real-time conversation and determining whether such action items have been completed. The system detects, during a meeting, a plurality of action items and an utterance that corresponds to a completed action item. Responsive to detecting the utterance, the system generates a similarity score with respect to a first action item of the plurality of action items. The system compares the similarity score to a first threshold. Responsive to determining that the similarity score does not exceed the first threshold, the system generates a second similarity score with respect to a second action item of the plurality of action items. The system compares the second similarity score to a second threshold, which exceeds the first threshold. Responsive to determining that the second similarity score exceeds the second threshold, the system marks the second action item as completed.

Abstract: Systems and methods for detecting a conflict between viewing selections of two users before viewing a media asset. In some aspects, the method comprises receiving an audio input through an audio channel, detecting a first utterance from a first user and a second utterance from a second user in the input of the audio channel, parsing the first utterance and the second utterance, analyzing the first utterance and the second utterance to determine context about the first media asset and the second media asset, and presenting a conflict to the users to display on the media asset.

Abstract: Disclosed is a digital pen that tracks the user's writing and provides useful feedback based on the user's writing. In one embodiment, the pen may provide feedback when the user has written a misspelled word, invalid mathematical expression, or any noncompliant expression. The pen may also provide feedback relating to the user's handwriting. The feedback may be visual, auditory, or tactile, and may be realtime or delayed. Statistics relating to the user's performance may be tracked, uploaded to external devices, and shared with others. This allows the user and interested parties to track the user's progress over time. The disclosed pen will be useful in educational settings.

Abstract: A voice-activated camera system for a computing device. The voice-activated camera system includes a processor, a camera module, a speech recognition module and a microphone for accepting user voice input. The voice-activated camera system includes authorized for only a specific user's voice, so that a camera function may be performed when the authorized user speaks the keyword, but the camera function is not performed when an unauthorized user speaks the keyword.

Abstract: Certain aspects of the present disclosure provide a method for performing voice activity detection, including: receiving audio data from an audio source of an electronic device; generating a plurality of model input features using a hardware-based feature generator based on the received audio data; providing the plurality of model input features to a hardware-based voice activity detection model; receiving an output value from the hardware-based voice activity detection model; and determining a presence of voice activity in the audio data based on the output value.