Abstract: Provided are an electronic device capable of providing text information corresponding to a user voice through a user interface and a method of controlling the electronic device. Specifically, an electronic device according to the present disclosure, when an image including at least one object is obtained, analyzes the image to identify the at least one object included in the image, and when a user voice is received, performs voice recognition on the user voice to obtain text information corresponding to the user voice, then identifies an object corresponding to the user voice among the at least one object included in the image, and displays a memo user interface (UI) including text information on an area corresponding to the object identified as corresponding to the user voice among areas on a display.

Abstract: A method includes connecting a call from a client device to a destination having an interactive voice response service; transcribing audio from the destination during the call to identify menu options of the interactive voice response service; generating visualizations representing the menu options; and outputting the visualizations to a display associated with the client device. A system includes a telephony system, an automatic speech recognition processing tool, and a visualization output generation tool. The telephony system connects a call from a client device to a destination having an interactive voice response service. The automatic speech recognition processing tool transcribes audio from the destination during the call to identify menu options of the interactive voice response service. The visualization output generation tool generates visualizations representing the menu options. The telephony system outputs the visualizations to a display associated with the client device.

Abstract: Techniques for recommending a skill experience to a user after a user-system dialog session has ended are described. Upon a dialog session ending, the system uses a first machine learning model to determine potential intents to recommend to a user. The system then uses a second machine learning model to determine a particular skill and intent to recommend. The system then prompts the user to accept the recommended skill and intent. If the user accepts, the system calls the recommended skill to execute. As part of calling the skill, the system sends to the skill at least one entity provided in a natural language user input of the ended dialog session. This enables the skill to skip welcome prompts, and initiate processing to output a response based on the intent and the at least one entity of the ended dialog session.

Abstract: A computer implemented method for the automated analysis or use of data is implemented by a voice assistant. The method comprises the steps of: (a) storing in a memory a structured, machine-readable representation of data that conforms to a machine-readable language (‘machine representation’); the machine representation including representations of user speech or text input to a human/machine interface; and (b) automatically processing the machine representations to analyse the user speech or text input.

Abstract: Disclosed herein are methods, systems, and computer-readable media for automatically generating and editing text. In an embodiment, a method may include receiving an input text prompt and receiving one or more user instructions. The method may also include accessing a language model based on the input text prompt and the one or more user instructions. The method may also include outputting, using the accessed language model, language model output text. The method may also include editing the input text prompt based on the language model and the one or more user instructions by replacing at least a portion of the input text prompt with the language model output text.

Abstract: Disclosed is a method of controlling an electronic apparatus. The method of controlling an electronic apparatus includes: displaying a screen including an input area configured to receive a text, receiving a speech and obtaining a text corresponding to the speech, performing a service operation corresponding to the input area by inputting the obtained text to the input area, and based on a result of performing the service operation, obtaining a plurality of similar texts including a similar pronunciation with the obtained text, and repeatedly performing the service operation by sequentially inputting the plurality of obtained similar texts to the input area.

Abstract: An example process includes: receiving a natural language input having a start time and including first and second portions respectively received from the start time to a first time and from the start time to a second time after the first time; determining an end time of the natural language input; executing, at least partially between the first time and the end time, a first task flow based on the first portion, including: obtaining a first executable object representing a first candidate action; executing, at least partially between the second time and the end time, a second task flow based on the second portion, including: obtaining a second executable object representing a second candidate action; in response to determining the end time, selecting a candidate action from a plurality of candidate actions each represented by a respective executable object; and executing the respective executable object representing the selected candidate action.

Abstract: In some implementations, an utterance that requests a voice call is received, the utterance is classified as spoken by a particular known user, the particular known user is determined to be associated with a personal voice number, and in response to determining that the particular known user is associated with a personal voice number, the voice call is initiated with the personal voice number.

Abstract: Systems and processes for operating an intelligent automated assistant are provided. For example, an intelligent automated assistant obtains a static vocabulary entry for an application and registers the static vocabulary entry to a knowledge base. While the application is running, the intelligent automated assistant receives a request from the application to register a dynamic vocabulary entry, and also registers the dynamic vocabulary entry. Upon receiving a user input, the intelligent automated assistant determines whether a matching vocabulary entry for the application has been registered and causes the application to perform a task based on the matching vocabulary entry.

Abstract: A method for dialogue processing, an electronic device and a storage medium are provided. The specific technical solution includes: obtaining a dialogue history; selecting a target machine from a plurality of machines; inputting the dialogue history into a trained dialogue model in the target machine to generate a response to the dialogue history, in which the dialogue model comprises a common parameter and a specific parameter, and different machines correspond to the same common parameter.

Abstract: A method, system and apparatus to generate an augmented voice command, including identifying a plurality of sounds from a respective plurality of transducers to a smart speaker device, generating a visualization of the sounds using an augmented reality device, wherein one or more of the sounds can be selected using the visualization, and generating the augmented voice command for the smart speaker device, wherein the augmented voice command comprises the one or more sounds selected using the visualization of the augmented reality device.

Abstract: Methods are provided, such as a method of interacting with a large language model (LLM), including the step of a processing system using a structured, machine-readable representation of data that conforms to a machine-readable language, such as a universal language, to provide new context data for the LLM, in order to improve the output, such as continuation text output, generated by the LLM in response to a prompt; and such as a method of interacting with a LLM, including the step of providing continuation data generated by the LLM to a processing system that uses a structured, machine-readable representation of data that conforms to a machine-readable language, such as a universal language, in which the processing system is configured to analyse the continuation output generated by the LLM in response to a prompt to enable an improved version of that continuation output to be provided to a user. Related computer systems are provided.

Abstract: Various embodiments discussed herein enable client applications to be heavily integrated with a voice assistant in order to perform commands associated with voice utterances of users via voice assistant functionality and also seamlessly cause client applications to automatically perform native functions as part of executing the voice utterance. Such heavy integration also allows particular embodiments to support multi-modal input from a user for a single conversational interaction. In this way, client application user interface interactions, such as clicks, touch gestures, or text inputs are executed alternative or in addition to the voice utterances.

Abstract: Systems and methods are provided for an application platform with flexible permissioning.

Abstract: The present disclosure including determining that a state of a load on an occupant in a vehicle on the basis of at least one of a traveling state of the vehicle (100), an external environment state of the vehicle, and a state of the occupant in the vehicle, and executing a dialogue with the occupant by executing a dialogue program corresponding to the state of the load on the occupant.

Abstract: Systems and methods for determining that artificial commands, in excess of a threshold value, are detected by multiple voice activated electronic devices is described herein. In some embodiments, numerous voice activated electronic devices may send audio data representing a phrase to a backend system at a substantially same time. Text data representing the phrase, and counts for instances of that text data, may be generated. If the number of counts exceeds a predefined threshold, the backend system may cause any remaining response generation functionality that particular command that is in excess of the predefined threshold to be stopped, and those devices returned to a sleep state. In some embodiments, a sound profile unique to the phrase that caused the excess of the predefined threshold may be generated such that future instances of the same phrase may be recognized prior to text data being generated, conserving the backend system's resources.

Abstract: Implementations herein relate to information describing one or more internal states of a technical system. Implementations herein are provided for characterizing reliability of various different third party servers, at least when reporting third party device statuses, as well as adapting protocols for device ecosystems affected by such reliability. Latency can affect accuracy of device states represented by assistant devices. Certain servers can be characterized as especially delayed when reporting an updated device state in response to a user request, and, as a result, the third party server can be correlated to a metric that characterizes the relative latency of the third party server. When the metric fails to satisfy a particular threshold, a server and/or client associated with the “ecosystem” of third party devices can affirmatively operate to retrieve device state updates, rather than passively await updates from a corresponding third party server.

Abstract: To provide a voice analysis system capable of performing voice recognition with higher accuracy. A voice analysis system including a first voice analysis terminal and a second voice analysis terminal, the first voice analysis terminal obtaining first conversation information, and the second voice analysis terminal obtaining second conversation information, wherein the voice analysis system comprises a conversation category selection unit which compares the number of related words included in the first conversation information and the number of related words included in the second conversation information, in each conversation category, and adopts the conversation category with the larger number of related words as a correct conversation category.

Abstract: According to an embodiment, an electronic device comprises a communication module, a memory, and a processor configured to, upon obtaining a first utterance related to a memory service, prepare to store first information for the memory service for the first utterance and store the first information including essential information, sensitivity information for the first information, and an authentication method for the first information, detected from the first utterance, in the memory, and obtain a second utterance for looking up information related to the memory service, upon identifying that the obtained second utterance is one for looking up the first information, complete authentication based on the authentication method, and provide the essential information by a providing method determined based on the sensitivity information. Various other embodiments may be provided.

Abstract: The present disclosure provides method and apparatus for generating a response in a human-machine conversation. A first sound input may be received in the conversation. A first audio attribute may be extracted from the first sound input, wherein the first audio attribute indicates a first condition of a user. A second sound input may be received in the conversation. A second audio attribute may be extracted from the second sound input, wherein the second audio attribute indicates a second condition of a user. A difference between the second audio attribute and the first audio attribute is determined, wherein the difference indicates a condition change of the user from the first condition to the second condition. A response to the second sound input is generated based at least on the condition change.

Abstract: The learning data generation device (10) of the present invention comprises: an end-of-talk predict unit (11) for performing: a first prediction in which it is predicted, based on utterance information on an utterance in the dialog, using the end-of-talk prediction model (16), whether the utterance is an end-of-talk utterance of the speaker; and a second prediction in which it is predicted, based on one or more prescribed rules, whether the utterance is an end-of-talk utterance; and a training data generate unit (13) for generating, when, in the first prediction it is predicted that the utterance is not an end-of-talk utterance and in the second prediction it is predicted that the utterance is an end-of-talk utterance, for the utterance information on the utterance, learning data to which training data indicating that the utterance is an end-of-talk utterance is appended.

Abstract: Examples of the present disclosure describe systems and methods for a windowing container that enables two or more windows associated with application(s) to be grouped within the container such that the windows may behave or function uniformly as a single window. For example, responsive to a request to group two windows, a container may be generated to include the windows arranged based on one or more rules and features enabling group functions associated with the container to be performed. When a group function is performed on the container, the function may be performed to each of the windows arranged therein as if they were a single window. As new windows are grouped within or existing windows are released from the container, the container and/or windows may be rearranged based on the rules. A state of the container may be stored to enable subsequent invocation of the container after closing.

Abstract: Systems and methods are provided for generating and operating a speech enhancement model optimized for generating noise-suppressed speech outputs for improved human listening and live captioning. A computing system obtains a speech enhancement model trained on a first training dataset to generate noise-suppressed speech outputs and an automatic speech recognition model trained on a second training dataset to generate transcription labels for spoken language utterances. A third training dataset comprising a set of spoken language utterances is applied to the speech enhancement model to obtain a first noise-suppressed speech output which is applied to the automatic speech recognition model to generate a noise-suppressed transcription output for the set of spoken language utterances.

Abstract: Methods, programming, and system for modifying a slot value are described herein. In a non-limiting embodiment, an intent may be determined based on a first utterance. A first slot-value pair may be obtained for the first utterance based on the intent, the first slot-value pair including a first slot and a first value associated with the first slot. A second value associated with the first slot may be identified, the second value being identified from a second utterance that was previously received. Based on the intent and the first slot, a type of update to be performed with respect to the second value may be determined. The second value may then be updated based on the first value and the type of update.

Abstract: Aspects of the disclosure relate to mobile device operations during an ongoing call. The mobile device operations may relate to payment processing operations. The mobile device may determine contact information associated with a party in an ongoing call. The mobile device may determine, based on the contact information, information associated with a payment transaction request. The mobile device may send, based on the determined information the payment transaction request. The mobile device may dynamically determine a user interface to be displayed during the call to facilitate the various mobile device operations.

Abstract: A system for assisting sharing of information includes circuitry to: input a plurality of sentences each representing a statement made by one of a plurality of users, the sentence being generated by speaking or writing during a meeting or by extracting from at least one of meeting data, email data, electronic file data, and chat data at any time; determine a statement type of the statement represented by each one of the plurality of sentences, the statement type being one of a plurality of statement types previously determined; select, from among the plurality of sentences being input, one or more sentences each representing a statement of a specific statement type of the plurality of types; and output a list of the selected one or more sentences as key statements of the plurality of sentences.

Abstract: A software-based system and method that provides a generalized scheme to voice-enable text-oriented chatbots. The system can be configured to adapt to a plurality of different types of chatbots, a plurality of different speech-to-text and text-to-speech services, a plurality of different grammars, and even a plurality of different languages. The system can further be configured to handle “HTTP complex” situations such as electronic calendars by automatically analyzing these HTTP complex situations into various sub-dialogs, which the system can then automatically use to communicate with users, and then present the final results back to the chatbot. These methods enable organizations to preserve their extensive investment in legacy chatbots while rapidly and relatively inexpensively providing voice functionality to a broader range of users.

Abstract: The present disclosure generally relates to methods and user interfaces for managing visual content at a computer system. In some embodiments, methods and user interfaces for managing visual content in media are described. In some embodiments, methods and user interfaces for managing visual indicators for visual content in media are described. In some embodiments, methods and user interfaces for inserting visual content in media are described. In some embodiments, methods and user interfaces for identifying visual content in media are described. In some embodiments, methods and user interfaces for translating visual content in media are described.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for natural language processing. One of the method includes receiving a voice input from a user device; generating a recognition output; receiving a user selection of one or more terms in the recognition output; receiving a user input of one or more letters replacing the user selected one or more terms; determining suggested correction candidates based in part on the user input and the voice input; and providing one or more suggested correction candidates to the user device as suggested corrected recognition outputs.

Abstract: Systems and methods are described herein for inserting dialogue into query responses by generating and using dialogue metadata in conjunction with response templates. Metadata for each portion of dialogue of a plurality of portions of dialogue from a number of content items is stored, including information regarding the source content item, a transcript of the dialogue, and grammatical information. Upon receiving a query related to a content item, a type of response is first determined. Based on the type of response, and using the dialogue metadata, a portion of dialogue is identified for insertion into the response. The identified portion of dialogue is retrieved and inserted at an appropriate position within the response. The response is then generated for output.

Abstract: A media edit point selection process can include a media editing software application programmatically converting speech to text and storing a timestamp-to-text map. The map correlates text corresponding to speech extracted from an audio track for the media clip to timestamps for the media clip. The timestamps correspond to words and some gaps in the speech from the audio track. The probability of identified gaps corresponding to a grammatical pause by the speaker is determined using the timestamp-to-text map and a semantic model. Potential edit points corresponding to grammatical pauses in the speech are stored for display or for additional use by the media editing software application. Text can optionally be displayed to a user during media editing.

Abstract: A computer executed process for mimicking human dialog, referred to herein as a “humanoid” or “humanoid system,” can be configured to provide automated customer support. The humanoid can identify a support issue for a customer, as well as a customer support campaign corresponding to the support issue. The humanoid can identify at least one machine learning model associated with the customer support campaign and can communicate with the customer using the at least one machine learning model. The humanoid can execute a support action to resolve the support issue.

Abstract: There is much data that is currently not being captured during user interaction with mobile apps that could provide insight into how to effectively address a user concern. Capturing such data may allow auto-adjustments of operational responses provided by mobile apps in response to detecting anomalous user inputs. Such anomalous user inputs may include keyboard dynamics or mobile device movement that deviate from an average or user specific levels. Such anomalous user inputs may indicate that a user concern is particularly urgent. Auto-adjustments to operation of a mobile app may include initiating targeted chatbot or live chat responses.

Abstract: An electronic device is provided. The electronic device includes a display, a communication circuit, a processor operatively connected to the display and the communication circuit, and a memory operatively connected to the processor. The memory stores instructions that, when executed, cause the processor to receive information about a time interval and user interface information, which are associated with a response to a user utterance input to a first external electronic device, from a second external electronic device through the communication circuit, to determine whether the display is in an active state within the time interval, and to provide a first user interface corresponding to the user interface information through the display based on the determination that the display is in the active state within the time interval.

Abstract: Techniques for decreasing (or eliminating) the possibility of a skill performing an action that is not responsive to a corresponding user input are described. A system may train one or more machine learning models with respect to user inputs, which resulted in incorrect actions being performed by skills, and corresponding user inputs, which resulted in the correct action being performed. The system may use the trained machine learning model(s) to rewrite user inputs that, if not rewritten, may result in incorrect actions being performed. The system may implement the trained machine learning model(s) with respect to ASR output text data to determine if the ASR output text data corresponds (or substantially corresponds) to previous ASR output text data that resulted in an incorrect action being performed.

Abstract: One or more computing devices, systems, and/or methods for user identification and authorization are provided. In an example, a voice command to perform an action is detected. A voice profile associated with a user is identified based upon voice characteristics of the voice command. In response to determining that the voice profile is not linked to an account associated with the action, the user is prompted for an identifier associated with a device for creating the account through the device. In response to receiving the identifier from the user, the identifier is utilized to facilitate creation of the account through the device.

Abstract: Methods, apparatus, systems, and computer-readable media for engaging an automated assistant to perform multiple tasks through a multitask command. The multitask command can be a command that, when provided by a user, causes the automated assistant to invoke multiple different agent modules for performing tasks to complete the multitask command. During execution of the multitask command, a user can provide input that can be used by one or more agent modules to perform their respective tasks. Furthermore, feedback from one or more agent modules can be used by the automated assistant to dynamically alter tasks in order to more effectively use resources available during completion of the multitask command.

Abstract: Systems and methods for correcting a voice query based on a subsequent voice query with a lower pronunciation rate. In some aspects, the systems and methods calculate first and second pronunciation rates of first and second voice queries. The systems and methods determine that the second pronunciation rate is lower than the first pronunciation rate and determine a first candidate pronunciation time for a first candidate word from the first voice query. The systems and methods determine a second candidate pronunciation time, adjusted to the first pronunciation rate, for the second candidate word from the second voice query. The systems and methods determine that the first candidate pronunciation time matches the second candidate pronunciation time and generate a third voice query based on the first voice query by replacing the first candidate word with the second candidate word.

Abstract: In a method for efficiently and accurately measuring the intelligibility of speech, a user may utter a sample text, and an automatic speech assessment (ASA) system may receive an acoustic signal encoding the utterance. An automatic speech recognition (ASR) module may generate an N-best output corresponding to the utterance and generate an intelligibility score representing the intelligibility of the utterance based on the N-best output and the sample text. Generating the intelligibility score may involve (1) calculating conditional intelligibility value(s) for the N recognition result(s), and (2) determining the intelligibility score based on the conditional intelligibility value of the most intelligible recognition result. Optionally, the process of generating the intelligibility score may involve adjusting the intelligibility score to account for environmental information (e.g., a pronunciation score for the user's speech and/or a confidence score assigned to the 1-best recognition result).

Abstract: A device may receive audio data based on a capturing of sounds associated with a structure. The device may obtain a model associated with the structure. The model may have been trained to receive the audio data as input, determine a score that identifies a likelihood that a sound is present in the audio data, and identify the sound based on the score. The device may determine at least one parameter associated with the sound. The device may generate a metric based on the at least one parameter associated with the sound, and perform an action based on the metric.

Abstract: A computer implemented method for the automated analysis or use of data is implemented by a voice assistant. The method comprises the steps of: (a) storing in a memory a structured, machine-readable representation of data that conforms to a machine-readable language (‘machine representation’); the machine representation including representations of user speech or text input to a human/machine interface; and (b) automatically processing the machine representations to analyse the user speech or text input.

Abstract: A method is for creating a controller for controlling an output unit from information in the context of medical diagnostics and therapy. The method includes providing a learning processing apparatus designed via an algorithm to recognize spoken words; providing on, or in, the learning processing apparatus, an untrained controller, designed to be trained via machine learning; providing a number of speech recordings, each including a communication during a medical procedure, wherein the speech recordings concern comparable medical procedures; performing a speech analysis of the speech recordings; and training the untrained controller according to a machine learning principle based upon the speech analysis of the speech recordings.

Abstract: A voice-operated system includes a processing device, and a controller that communicates with the processing device. The processing device includes a first processor to perform: displaying, on an operating panel, an operation screen for instructing a process for execution by the processing device, to receive user's instruction; and executing a process corresponding to a command received from the controller. The controller includes a second processor to perform: generating the command for the processing device based on an input voice; and transmitting the command to the processing device. The generation of the command includes, when the voice instructs the processing device to execute a first process, generating a first command for instructing the operating panel to display the operation screen for instructing execution of the first process; and when the voice instructs the processing device to execute a second process, generating a second command for instructing execution of the second process.

Abstract: Systems and methods of generating voice-based software applications are provided. A system can receive, from an application developer computing device, a request to build a voice-based software application. The system can select an application template from a plurality of application templates. The selected application template can include a module that corresponds to a function of the voice-based software application. The system can provide the selected application template to the application developer computing device. The system can receive, from the application developer computing device, an input for a field of the at least one module of the selected application template. The system can generate the voice-based software application based on the selected application template and the input for the at least one field of the at least one module of the selected application template.

Abstract: Implementations can receive audio data corresponding to a spoken utterance of a user, process the audio data to generate a plurality of speech hypotheses, determine an action to be performed by an automated assistant based on the speech hypotheses, and cause the computing device to render an indication of the action. In response to the computing device rendering the indication, implementations can receive additional audio data corresponding to an additional spoken utterance of the user, process the additional audio data to determine that a portion of the spoken utterance is similar to an additional portion of the additional spoken utterance, supplant the action with an alternate action, and cause the automated assistant to initiate performance of the alternate action. Some implementations can determine whether to render the indication of the action based on a confidence level associated with the action.

Abstract: Methods, apparatus, systems and articles of manufacture for recognizing speech are disclosed. An example system includes one or more processors to execute instructions to: identify a plurality of phonemes in a speech signal; perform a comparison of a subset of the phonemes to a phonetic string, the phonetic string representative of at least a portion of a wake up phrase; determine if one or more of the phonemes of the subset correspond to the wake up phrase based on the comparison; and generate a hypothesis of a command included in the speech signal by excluding the wake up phrase when one or more of the phonemes of the subset correspond to the wake up phrase or a portion of the wake up phrase.

Abstract: A computer system may connect to various customer-facing devices and manage or automate the order process between a retail store and the customer. The computer system may perform the dialogue and receive an order for items from the retail store and may perform quality control monitoring of the dialogue between customers and employees taking orders. The ordering system may utilize the ordered items in combination with various contextual cues to determine a customer identity which may then be linked to past orders and/or various order preferences. Based on the determined customer identity, the system may provide recommendations of additional order items or order alterations to the customer before personally identifying information has been collected from the customer. The determination of the customer identity and the determination of recommendations may be performed by machine learning algorithms that were trained on customer data and the retail store products.

Abstract: The present invention relates to a drive for an electric application such as an electric motor, said drive including at least one microphone for registering noise signals occurring at the drive, wherein the microphone is connectable to a computing device for analysing the registered noise signals. The registered noise signals may be used for a maintenance process of the drive and/or a fine-tuning process of a drive control method of the drive. The present invention also relates to a maintenance process, in particular a predictive maintenance process for a corresponding drive. Furthermore, the present invention relates to a process for fine tuning a drive control method of a corresponding drive.

Abstract: A device and a method of providing a to-do list of a user are provided. The device includes a controller configured to collect behavior information about behavior between the user and another user, the behavior being performed by using the device, generate a to-do list of the user based on the collected behavior information, and determine an unperformed task not performed by the user from among at least one task in the to-do list by using log information about an operation of the device, and an output unit configured to output notification information in a dialogue style, along with a notification reason for notifying the determined unperformed task.

Abstract: An example emergency vehicle siren system can include: one or more emergency lights; a siren; and a controller including a processor and memory, the memory encoding instructions which, when executed by the processor, cause the controller to modify a sound of the siren based upon a context of the vehicle, the context including at least one of a speed and location of the vehicle.