Abstract: The disclosure generally relates to methods and systems for complex natural language task understanding for embodied robots or agents. Conventional works on relation extraction generally find relevant triplets in a natural language phrase, but neither ground the task nor ground the arguments. The present disclosure implements a Grounded Argument and Task Extraction (GATE) technique that extracts a set of tasks and relevant arguments from the complex natural language instruction. The GATE uses an encoder-decoder neural network with nested decoding technique. The extracted tasks are mapped (grounded) to the known skill set of the robot and arguments are mapped (grounded) to objects within the environment, classifies the tokens as many times as possible which existing sequence labeling cannot do.

Abstract: Implementations disclosed herein are directed to unsupervised federated training of global machine learning (“ML”) model layers that, after the federated training, can be combined with additional layer(s), thereby resulting in a combined ML model. Processor(s) can: detect audio data that captures a spoken utterance of a user of a client device; process, using a local ML model, the audio data to generate predicted output(s); generate, using unsupervised learning locally at the client device, a gradient based on the predicted output(s); transmit the gradient to a remote system; update weight(s) of the global ML model layers based on the gradient; subsequent to updating the weight(s), train, using supervised learning remotely at the remote system, a combined ML model that includes the updated global ML model layers and additional layer(s); transmit the combined ML model to the client device; and use the combined ML model to make prediction(s) at the client device.

Abstract: A method and a voice assistant device for managing private data are provided. The voice assistant device includes a processor configured to detect a privacy triggering event while obtaining at least one voice input from a first user in a voice input mode, switch from the voice input mode to a non-voice input mode in response to the privacy triggering event, obtain a non-voice input from the first user in the non-voice input mode, and execute an operation of the voice assistant device corresponding to the non-voice input.

Abstract: Systems and methods are described herein for locally interpreting a voice query and for managing a storage size of data stored locally to support such local interpretation of voice queries. A voice query is received and compared with a plurality of stored voice queries having similar audio characteristics. If a match is identified, text corresponding to the matching stored voice query is retrieved, and an action corresponding to the retrieved text is performed. If the locally stored table does not contain a stored voice query that matches the voice query, the voice query is transmitted to a remote server for transcription. Once the transcription is received from the remote server, the voice query and the transcription are stored in the table in association with one another.

Abstract: Systems and techniques for are described herein. A voice profile may be generated for a user. An audio stream may be received including an authentication voice of the user. It may be determined that the authentication voice does not match a first set of authentication criteria. The audio stream may be compared to a second set of authentication criteria. The user may be authenticated based on the comparison.

Abstract: The disclosure herein describes a system and method for attentive sentence similarity scoring. A distilled sentence embedding (DSE) language model is trained by decoupling a transformer language model using knowledge distillation. The trained DSE language model calculates sentence embeddings for a plurality of candidate sentences for sentence similarity comparisons. An embedding component associated with the trained DSE language model generates a plurality of candidate sentence representations representing each candidate sentence in the plurality of candidate sentences which are stored for use in analyzing input sentences associated with queries or searches. A representation is created for the selected sentence. This selected sentence representation is used with the plurality of candidate sentence representations to create a similarity score for each candidate sentence-selected sentence pair.

Abstract: Techniques for constructing and otherwise managing knowledge graphs in information processing system environments are disclosed. For example, a method comprises the following steps. The method collects data from a plurality of data sources. The method extracts structured data and unstructured data from the collected data, wherein unstructured data is extracted using an unsupervised machine learning process. The method forms a plurality of sub-graph structures comprising a sub-graph structure for each of the data sources based on at least a portion of the extracted structured data and unstructured data. The method combines the plurality of sub-graph structures to form a combined graph structure representing the collected data from the plurality of data sources. The resulting combined graph structure is a comprehensive knowledge graph.

Abstract: The disclosed computer-implemented method includes analyzing, by a speech detection system, a media file to detect lip movement of a speaker who is visually rendered in media content of the media file. The method additionally includes identifying, by the speech detection system, audio content within the media file, and improving accuracy of a temporal correlation of the speech detection system. The method may involve correlating the lip movement of the speaker with the audio content, and determining, based on the correlation between the lip movement of the speaker and the audio content, that the audio content comprises speech from the speaker. The method may further involve recording, based on the determination that the audio content comprises speech from the speaker, the temporal correlation between the speech and the lip movement of the speaker as metadata of the media file. Various other methods, systems, and computer-readable media are disclosed.

Abstract: A method of presenting a signal to a speech processing engine is disclosed. According to an example of the method, an audio signal is received via a microphone. A portion of the audio signal is identified, and a probability is determined that the portion comprises speech directed by a user of the speech processing engine as input to the speech processing engine. In accordance with a determination that the probability exceeds a threshold, the portion of the audio signal is presented as input to the speech processing engine. In accordance with a determination that the probability does not exceed the threshold, the portion of the audio signal is not presented as input to the speech processing engine.

Abstract: A system and method for recognizing (reading) the tongue movements, vocalizations, and throat vibrations of a person and converting (translating) them into meaningful synthesized words, which could be pronounced by an electronic speaker and/or displayed on a display. Often a patient/person who has lost the ability to speak may still be able to move their tongues, or make unfathomable sounds, which cannot be recognized as intelligible words. The system and method can continuously record the movement of the patient's tongue, vocalizations, and throat sounds and extract small video segments corresponding to different words attempted by the patient. Each of these video segments can then be analyzed by AI software or other configured software to match the specific tongue movement with a pre-learned reference word, and once identified, the computer/system can speak or verbalize the word, and/or display it on a screen.

Abstract: Disclosed are a method, a device, and a program for selecting a main speaker among speakers included in a sound source or a conversation record based on the sound source or the conversation record including conversation contents of at least one speaker and generating a summary based on the main speaker. A method of generating a summary for a sound source, the method being performed by at least one computing device, includes: generating a speak score for at least one speaker based on the sound source; determining a main speaker of the sound source based on a speak score for said at least one speaker; and generating a summary for the sound source in consideration of the determined main speaker.

Abstract: Methods, apparatus, and computer readable media are described related to automated assistants that proactively incorporate, into human-to-computer dialog sessions, unsolicited content of potential interest to a user. In various implementations, in an existing human-to-computer dialog session between a user and an automated assistant, it may be determined that the automated assistant has responded to all natural language input received from the user. Based on characteristic(s) of the user, information of potential interest to the user or action(s) of potential interest to the user may be identified. Unsolicited content indicative of the information of potential interest to the user or the action(s) may be generated and incorporated by the automated assistant into the existing human-to-computer dialog session.

Abstract: Various embodiments provide methods, apparatus, systems, computing entities, and/or the like, generating predictions based at least in part on recognizing significant words in unstructured text. In an embodiment, a method is provided. The method comprises: generating a plurality of word-level tokens for an input unstructured textual data object; and for each word-level token: determining a significance type and a significance subtype for the word-level token by using a significance recognition machine learning model, and assigning a significance token label or an insignificance token label to the word-level token.

Abstract: A computer-implemented system and method relate to natural language processing and knowledge representation and reasoning. A first dataset is created that includes input data and situational data. The situational data provides context for the input data. An encoder is configured to generate an encoded representation of the first dataset. The encoder includes at least an encoding network of a first pre-trained generative machine learning model, which relates to a generative knowledge graph. A decoder includes a decoding network of a second pre-trained generative machine learning model. The decoder is configured to generate response data based on the first dataset by decoding the encoded representation. The decoder is also configured to generate event-centric knowledge based on the first dataset by decoding the encoded representation. The input data and the response data are connected to the same event-centric knowledge via the generative knowledge graph.

Abstract: A method and device for encoding a stereo sound signal comprise stereo encoders using stereo modes operating in time domain (TD), in frequency domain (FD) or in modified discrete Fourier transform (MDCT) domain. A controller controls switching between the TD, FD and MDCT stereo modes. Upon switching from one stereo mode to the other, the switching controller may (a) recalculate at least one length of down-processed/mixed signal in a current frame of the stereo sound signal, (b) reconstruct a down-processed/mixed signal and also other signals related to the other stereo mode in the current frame, (c) adapt data structures and/or memories for coding the stereo sound signal in the current frame using the other stereo mode, and/or (d) alter a TD stereo channel down-mixing to maintain a correct phase of left and right channels of the stereo sound signal. Corresponding stereo sound signal decoding method and device are described.

Abstract: In one aspect, an example method includes (i) obtaining media, wherein the obtained media includes (a) audio representing speech and (b) video; (ii) using at least the audio representing speech as a basis to generate speech text; (iii) using at least the audio representing speech to determine starting and ending time points of the speech; and (iv) using at least the generated speech text and the determined starting and ending time points of the speech to (a) generate closed-captioning or subtitle data that includes closed-captioning or subtitle text based on the generated speech text and (b) associating the generated closed-captioning or subtitle data with the obtained media, such that the closed-captioning or subtitle text is time-aligned with the video based on the determined starting and ending time points of the speech.

Abstract: An operation support apparatus includes: an acquisition unit that acquires an operation status of transportation means; a generation unit that performs processing of recognizing voices of a first staff who gives an instruction to operate the transportation means and a second staff who is instructed by the first staff, and generates character information obtained by converting the recognized voice into characters; a detection unit that performs syntax analysis on the character information and detects wrong recognition by the first or second staff; and a display control unit that displays the character information and the detection result of the wrong recognition by the detection unit on a display device visually recognizable by the first staff, thereby reducing occurrence of an accident due to the wrong recognition by the staff related to operation of the transportation means at a site where the transportation means is operated.

Abstract: A method, performed by an electronic device, of providing a conversational service includes: receiving an utterance input; identifying a temporal expression representing a time in a text obtained from the utterance input; determining a time point related to the utterance input based on the temporal expression; selecting a database corresponding to the determined time point from among a plurality of databases storing information about a conversation history of a user using the conversational service; interpreting the text based on information about the conversation history of the user, the conversation history information being acquired from the selected database; generating a response message to the utterance input based on a result of the interpreting; and outputting the generated response message.

Abstract: Implementations set forth herein relate to phasing-out of vehicle computing device versions while ensuring useful responsiveness of any vehicle computing device versions that are still in operation. Certain features of updated computing devices may not be available to prior versions of computing devices because of hardware limitations. The implementations set forth herein eliminate crashes and wasteful data transmissions caused by prior versions of computing devices that have not been, or cannot be, upgraded. A server device can be responsive to a particular intent request provided to a vehicle computing device, despite the intent request being associated with an action that a particular version of the vehicle computing device cannot execute. In response, the server device can elect to provide speech to text data, and/or natural language understanding data, in furtherance of allowing the vehicle computing device to continue leveraging resources at the server device.

Abstract: An electronic apparatus and a controlling method thereof are provided. The electronic apparatus includes a memory storing an artificial intelligence model, a display configured to display a keypad screen, and a processor configured to, based on a swipe interaction contacting a first key and connecting a plurality of keys being on the keypad screen, remove from the input window, a first word that is pre-input to the input window, obtain a word candidate list corresponding to the swipe interaction by inputting information regarding a trajectory of the swipe interaction to an artificial intelligence model, and control the display to display, in the input window, a second word having a highest score in the word candidate list.