Abstract: An approach for automatically generate the Natural Language Interface (NLI) directly from the Graphical User Interface (GUI) code is disclosed. The approach leverages the use of mapping between GUI components to pre-defined NLI components in order to generate the necessary NLI components (e.g., intent example, entities, etc.) from the GUI code representation. The approach can leverage pre-defined patterns in order to generate these intent examples for each kind of NLI components. The created NLI dialog can be used simultaneously with the GUI or as a standalone feature.

Abstract: A method including searching, on the basis of a voiceprint feature of a speaker, for an identifier of the speaker in a speaker registry, the voiceprint feature of the speaker being a parameter obtained according to a voice signal of the speaker captured by a microphone array; if position information corresponding to the identifier of the speaker in the speaker registry is different from position information of the speaker, updating the speaker registry, the position information of the speaker being a parameter obtained according to the voice signal of the speaker captured by the microphone array; and labeling the voice signal of the speaker with the identifier of the speaker, so as to track the speaker. The present disclosure enables voice tracking of multiple persons.

Abstract: System and method for generating disambiguated terms in automatically generated transcripts and employing the system are disclosed. Exemplary implementations may: obtain a set of transcripts representing various speech from users; obtain indications of correlated correct and incorrect transcripts of spoken terms; use a vector generation model to generate vectors for individual instances of the correctly transcribed terms and individual instances the incorrectly transcribed terms based on text and contexts of the individual transcribed terms; and train the vector generation model to reduce spatial separation of the vectors generated for the spoken terms in the correlated correct transcripts and the incorrect transcripts.

Abstract: An electronic device for predicting an intention of a user is configured to provide a question to the user and predict at least one first intention of the user based on context information associated with the user. The device is also configured to determine a question based on the at least one first intention of the user. The device is further configured to provide the question to the user. The device is additionally configured to receive a response to the question from the user. The device is also configured to predict at least one second intention of the user based on the at least one first intention of the user and the response to the question from the user.

Abstract: An electronic apparatus and a control method thereof are provided. A method of controlling an electronic apparatus according to an embodiment of the disclosure includes: receiving input of a first utterance, identifying a first task for the first utterance based on the first utterance, providing a response to the first task based on a predetermined response pattern, receiving input of a second utterance, identifying a second task for the second utterance based on the second utterance, determining the degree of association between the first task and the second task, and setting a response pattern for the first task based on the second task based on the determined degree of association satisfying a predetermined condition. The control method of an electronic apparatus may use an artificial intelligence model trained according to at least one of machine learning, a neural network, or a deep learning algorithm.

Abstract: Systems, apparatuses, and methods are described for a privacy blocking device configured to prevent receipt, by a listening device, of video and/or audio data until a trigger occurs. A blocker may be configured to prevent receipt of video and/or audio data by one or more microphones and/or one or more cameras of a listening device. The blocker may use the one or more microphones, the one or more cameras, and/or one or more second microphones and/or one or more second cameras to monitor for a trigger. The blocker may process the data. Upon detecting the trigger, the blocker may transmit data to the listening device. For example, the blocker may transmit all or a part of a spoken phrase to the listening device.

Abstract: A computer-implemented method, comprising, by an artificial-reality (AR) design tool: receiving, through a user interface (UI) of the AR design tool, instructions to add a voice-command module to an AR effect, the voice-command module having an intent type and at least one slot, the slot associated with one or more entities; establishing, according to instructions received through the UI, a logical connection between the slot and a logic module configured to generate the AR effect depending on a runtime value associated with the slot; and generate, for the AR effect, an executable program configured to: determine that a detected utterance corresponds to the intent type and includes one or more words associated with the slot; select, based on the one or more words, one of the one or more entities as the runtime value for the slot; send the runtime value to the logic module according to the logical connection.

Abstract: A system for audio-visual multi-speaker speech separation. The system includes a processing circuitry and a memory containing instructions that, when executed by the processing circuitry, configure the system to: receive audio signals captured by at least one microphone; receive video signals captured by at least one camera; and apply audio-visual separation on the received audio signals and video signals to provide isolation of sounds from individual sources, wherein the audio-visual separation is based, in part, on angle positions of at least one speaker relative to the at least one camera. The system provides for reliable speech processing and separation in noisy environments and environments with multiple users.

Abstract: A device includes one or more processors configured to receive an audio data sample and to provide the audio data sample to a dynamic classifier. The dynamic classifier is configured to generate a classification output corresponding to the audio data sample. The one or more processors are further configured to selectively access a particular device based on the classification output.

Abstract: A method, computer program product, and computing system for defining model representative of a plurality of acoustic variations to a speech signal, thus defining a plurality of time-varying spectral modifications. The plurality of time-varying spectral modifications may be applied to a plurality of feature coefficients of a target domain of a reference signal, thus generating a plurality of time-varying spectrally-augmented feature coefficients of the reference signal.

Abstract: A method and apparatus for controlling learning of a model for estimating an intention of an input utterance is disclosed. A method of controlling learning of a model for estimating an intention of an input utterance among a plurality of intentions includes providing a first index corresponding to the number of registered utterances for each intention, providing a second index corresponding to a learning level for each intention, providing a learning target setting interface such that at least one intention that is to be a learning target is selected from among the intentions based on the first index and the second index, and training the model based on the registered utterances for each intention and setting of the learning target for each intention.

Abstract: A method for adapting hotword recognition includes receiving audio data characterizing a hotword event detected by a first stage hotword detector in streaming audio captured by a user device. The method also includes processing, using a second stage hotword detector, the audio data to determine whether a hotword is detected by the second stage hot word detector in a first segment of the audio data. When the hotword is not detected by the second stage hotword detector, the method includes, classifying the first segment of the audio data as containing a negative hotword that caused a false detection of the hotword event in the streaming audio by the first stage hotword detector. Based on the first segment of the audio data classified as containing the negative hotword, the method includes updating the first stage hotword detector to prevent triggering the hotword event in subsequent audio data that contains the negative hotword.

Abstract: Systems and methods are provided for training a machine learning model to learn speech representations. Labeled speech data or both labeled and unlabeled data sets is applied to a feature extractor of a machine learning model to generate latent speech representations. The latent speech representations are applied to a quantizer to generate quantized latent speech representations and to a transformer context network to generate contextual representations. Each contextual representation included in the contextual representations is aligned with a phoneme label to generate phonetically-aware contextual representations. Quantized latent representations are aligned with phoneme labels to generate phonetically aware latent speech representations.

Abstract: A method for conducting a conversation between a user and a virtual agent is disclosed. The method includes receiving, by an ASR sub-system, a plurality of utterances from the user, and converting, by the ASR sub-system, each utterance of the plurality of utterances into a text message. The method further includes determining, by a NLU sub-system, an intent, at least one entity associated to the intent, or a combination thereof from the text message.

Abstract: A device for calculating cardiovascular heartbeat information is configured to receive an electronic audio signal with information representative of a human voice signal in the time-domain, the human voice signal comprising a vowel audio sound of a certain duration and a fundamental frequency; generate a power spectral profile of a section of the electronic audio signal, and detect the fundamental frequency (F0) in the generated power spectral profile; filter the received audio signal within a band around at least the detected fundamental frequency (F0) and thereby generating a denoised audio signal; generate a time-domain intermediate signal that captures frequency, amplitude and/or phase of the denoised audio signal; detect and calculate heartbeat information within a human cardiac band in the intermediate signal.

Abstract: A syntactic analysis apparatus according to an embodiment of the present disclosure may include an input device receiving a phrase uttered from a user, and a learning device performing at least one or more of extension of an intent output layer for classifying an utterance intent of the user from the uttered phrase and extension of a slot output layer for classifying a slot including information of the phrase and extending a pre-generated utterance syntactic analysis model, such that the uttered phrase is classified into the extended intent output layer and the extended slot output layer, thereby broadly classifying an intent and a slot for the phrase uttered from a user.

Abstract: Processor(s) of a client device can: identify a textual segment stored locally at the client device; process the textual segment, using a speech synthesis model stored locally at the client device, to generate synthesized speech audio data that includes synthesized speech of the identified textual segment; process the synthesized speech, using an on-device speech recognition model that is stored locally at the client device, to generate predicted output; and generate a gradient based on comparing the predicted output to ground truth output that corresponds to the textual segment. In some implementations, the generated gradient is used, by processor(s) of the client device, to update weights of the on-device speech recognition model. In some implementations, the generated gradient is additionally or alternatively transmitted to a remote system for use in remote updating of global weights of a global speech recognition model.

Abstract: Disclosed embodiments include a network device having a split network stack that includes a physical (PHY) layer associated with first and second media access control (MAC) protocol sublayers, a processing device, and memory storing instructions that, when executed by the processing device, cause the processing device to select a route through the split network stack that includes one of the first and second MAC protocol sublayers but not the other one of the first and second MAC protocol sublayers.

Abstract: Artificial assistant system notification techniques are described that overcome the challenges of conventional search techniques. In one example, a user profile is generated to describe aspects of products or services learned through natural language conversations between a user and an artificial assistant system. These aspects may include price as well as non-price aspects such as color, texture, material, and so forth. To learn the aspects, the artificial assistant system may leverage spoken utterances and text initiated by the user as well as learn the aspects from digital images output as part of the conversation. Once generated, the user profile is then usable by the artificial assistant system to assist in subsequent searches.

Abstract: Providing contextual help in an interactive voice system includes receiving a plurality of user interaction events during a user interaction window, wherein each of the user interaction events comprises one of a low quality voice transcription event from a speech-to-text (STT) service or a no-intent matching event from a natural language processing (NLP) service and receiving a respective transcription confidence score from the STT service for each of the plurality of user interaction events. For a one of the plurality of user interaction events, a determination is made of how to respond to a user providing the user interaction events based on how many events comprise the plurality of events and the transcription confidence score for the one event; and then instructions are provided to cause the determined response to be presented to the user in accordance with the determination of how to respond.

Abstract: Intent determination based on one or more multi-model structures can include generating an output from each of a plurality of domain-specific models in response to a received input. The domain-specific models can comprise simultaneously trained machine learning models that are trained using a corresponding local loss metric for each domain-specific model and a global loss metric for the plurality of domain-specific models. The presence or absence of an intent corresponding to one or more domain-specific models can be determined by classifying the output of each domain-specific model.

Abstract: A method for expanding an initial ontology via processing of communication data, wherein the initial ontology is a structural representation of language elements comprising a set of entities, a set of terms, a set of term-entity associations, a set of entity-association rules, a set of abstract relations, and a set of relation instances. A method for extracting a set of significant phrases and a set of significant phrase co-occurrences from an input set of documents further includes utilizing the terms to identify relations within the training set of communication data, wherein a relation is a pair of terms that appear in proximity to one another.

Abstract: Methods and systems for training and/or using a language selection model for use in determining a particular language of a spoken utterance captured in audio data. Features of the audio data can be processed using the trained language selection model to generate a predicted probability for each of N different languages, and a particular language selected based on the generated probabilities. Speech recognition results for the particular language can be utilized responsive to selecting the particular language of the spoken utterance. Many implementations are directed to training the language selection model utilizing tuple losses in lieu of traditional cross-entropy losses. Training the language selection model utilizing the tuple losses can result in more efficient training and/or can result in a more accurate and/or robust model—thereby mitigating erroneous language selections for spoken utterances.

Abstract: The disclosure herein describes determining topics of communication transcripts using trained summarization models. A first communication transcript associated with a first communication is obtained and divided into a first set of communication segments. A first set of topic descriptions is generated based on the first set of communication segments by analyzing each communication segment of the first set of communication segments with a generative language model. A summarization model is trained using the first set of communication segments and associated first set of topic descriptions as training data. The trained summarization model is then applied to a second communication transcript and, based on applying the trained summarization model to the second communication transcript, a second set of topic descriptions of the second communication transcript is generated.

Abstract: A computerized method for voice authentication of a customer in a self-service system is provided. A request for authentication of the customer is received and the customer is enrolled in the self-service system with a text-independent voice print. A passphrase from a plurality of passphrases to transmit to the customer is determined based on comparing each of the plurality of passphrases to a text-dependent or text-independent voice biometric model. The passphrase is transmitted to the customer, and when the customer responds, an audio stream of the passphrase is received. The customer is authenticated by comparing the audio stream of the passphrase against the text-independent voice print. If the customer is authenticated, then the audio stream of the passphrase and the topic of the passphrase may be stored.

Abstract: In some cases, one or more heuristics can be automatically generated using a small dataset of segments previously labeled by one or more domain experts. The generated one or more heuristics along with one or more patterns can be used to assign training labels to a large unlabeled dataset of segments. A subset of segments representing an occurrence of verbal harassment can be selected using the assigned training labels. Randomly selected segments can be used as being indicative of a non-occurrence of verbal harassment. The selected subset of segments and randomly selected segments can be used to train one or more machine learning models for verbal harassment detection.

Abstract: System and method for generating disambiguated terms in automatically generated transcriptions including instructions within a knowledge domain and employing the system are disclosed.

Abstract: The present invention discloses a smart counting method and system in manufacturing, specifically in custom clothing or fabric manufacturing. The smart counting method and system uses a camera to feed real-time image data of a working platform where a worker takes a unfinished clothing or fabric, processes the clothing or fabric, and puts the finished clothing or fabric in a finished pile to a processing unit. The processing unit automatically starts a new work order and counts the number of finished products in this work order by using computer vision techniques.

Abstract: According to one embodiment, a recognition device includes storage and a processor. The storage is configured to store a first recognition model, a first data set, and tags, for each first recognition model. The processor is configured to acquire a second data set, execute recognition processing of the second recognition target data in the second data set by using the first recognition model, extract a significant tag of the tags stored in the storage in association with the first recognition model, based on the recognition processing result and the second correct data in the second data set, and create a second recognition model based on the acquired second data set and the first data set stored in the storage in association with the extracted tag.

Abstract: Methods, systems, and computer program products for unsupervised tunable stylized text transformations are provided herein. A computer-implemented method includes identifying amendable portions of input text by processing at least a portion of the input text using at least one neural network; determining stylistic text modifications to the amendable portions of the input text, the text modifications encompassing a set of stylistic parameters, wherein said determining comprises processing at least a portion of the set of stylistic parameters using at least one neural network; generating a stylized output set of text by transforming at least a portion of the input text, wherein said transforming comprises modifying at least one of the amendable portions of the input text via at least one of the stylistic text modifications encompassed by the set of stylistic parameters; and outputting the stylized output set of text to at least one user.

Abstract: Systems and methods of returning location and/or event results using information mined from non-textual information are provided. Non-textual information is captured using a hardware component of a user device. Text-based social media content input on the user device is then retrieved. A location of the user device is determined using a global positioning system module in the user device. The non-textual information is converted to a machine-analyzable format, and the converted non-textual information is compared to a database of converted non-textual information samples to analyze and classify the converted non-textual information. The classification is sent to a server for storage in a database in a manner that ties the classification to the geographical location of the user device.

Abstract: A system includes acquisition of a domain grammar, determination of an interpolated grammar based on the domain grammar and a base grammar, determination of a delta domain grammar based on an augmented first grammar and the interpolated grammar, determination of an out-of-vocabulary class based on the domain grammar and the base grammar, insertion of the out-of-vocabulary class into a composed transducer composed of the augmented first grammar and one or more other transducers to generate an updated composed transducer, composition of the delta domain grammar and the updated composed transducer, and application of the composition of the delta domain grammar and the updated composed transducer to an output of an acoustic model.

Abstract: A speech recognition method includes segmenting captured voice information to obtain a plurality of voice segments, and extracting voiceprint information of the voice segments; matching the voiceprint information of the voice segments with a first stored voiceprint information to determine a set of filtered voice segments having voiceprint information that successfully matches the first stored voiceprint information; combining the set of filtered voice segments to obtain combined voice information, and determining combined semantic information of the combined voice information; and using the combined semantic information as a speech recognition result when the combined semantic information satisfies a preset rule.

Abstract: Systems, methods performed by data processing apparatus and computer storage media encoded with computer programs for receiving an utterance from a user in a multi-user environment, each user having an associated set of available resources, determining that the received utterance includes at least one predetermined word, comparing speaker identification features of the uttered predetermined word with speaker identification features of each of a plurality of previous utterances of the predetermined word, the plurality of previous predetermined word utterances corresponding to different known users in the multi-user environment, attempting to identify the user associated with the uttered predetermined word as matching one of the known users in the multi-user environment, and based on a result of the attempt to identify, selectively providing the user with access to one or more resources associated with a corresponding known user.

Abstract: Example embodiments provide techniques for configuring a natural-language processing system to perform a new function given at least one sample invocation of the function. The training data consisting of the sample invocation may be augmented by determining which subset of available training data most closely resembles the sample invocation and/or function. The effect of re-training a component this this augmented training data may be determined, and an annotator may review any annotations corresponding to the invocation if the effect is large.

Abstract: An electronic device includes: a camera; a microphone; a display; a memory; and a processor configured to receive an input for activating an intelligent agent service from a user while at least one application is executed, identify context information of the electronic device, control to acquire image information of the user through the camera, based on the identified context information, detect movement of a user's lips included in the acquired image information to recognize a speech of the user, and perform a function corresponding to the recognized speech.

Abstract: A system and method for assisting communication through predictive speech is provided. A database includes commonly used words, phrases, and images, each associated with at least one context cue. A processor is configured to determine the user's context and provide a number of possible initial phrases associated with the determined context. Further words, phrases, or images are provided following, and based at least in part on, user selection of the initial phrase. The selected initial phrase and the subsequently selected words, phrases, and images are communicated.

Abstract: A method of providing a conversational agent for interacting with a user may include declaratively defining a task model of a task using a task modelling language, storing the task model in a computer-readable storage medium, generating a natural language grammar based on the task model, storing the natural language grammar in the computer-readable storage medium, receiving a user input from the user, interpreting the user input with a processor based on the task model and the natural language grammar, generating an agent response to the user input with the processor based on the task model, and communicating the agent response to the user.

Abstract: An agent automation system includes a memory configured to store a corpus of utterances and a semantic mining framework and a processor configured to execute instructions of the semantic mining framework to cause the agent automation system to perform actions, wherein the actions include: detecting intents within the corpus of utterances; producing intent vectors for the intents within the corpus; calculating distances between the intent vectors; generating meaning clusters of intent vectors based on the distances; detecting stable ranges of cluster radius values for the meaning clusters; and generating an intent/entity model from the meaning clusters and the stable ranges of cluster radius values, wherein the agent automation system is configured to use the intent/entity model to classify intents in received natural language requests.

Abstract: Memory deterioration detection and evaluation includes capturing human utterances with a voice interface and generating, for a user, a human utterances corpus that comprises human utterances selected from the plurality of human utterances based on meanings of the human utterances as determined by natural language processing by a computer processor. Based on data generated in response to signals sensed by one or more sensing devices operatively coupled with the computer processor, contextual information corresponding to one or more human utterances of the corpus is determined. Patterns among the corpus of human utterances are recognized based on pattern recognition performed by the computer processor using one or more machine learning models. Based on the pattern recognition a change in memory functioning of the user is identified. The identified change is classified, based on the contextual information, as to whether the change is likely due to memory impairment of the user.

Abstract: A method for generating a set of concepts related to a target concept includes accessing a set of candidate concepts, embedding the target concept and the set of candidate concepts in a semantic vector space, selecting one or more intermediate concepts from the set of candidate concepts in response to determining whether each embedded candidate concept in the set of embedded candidate concepts satisfies a predetermined relationship with the embedded target concept, and filtering the one or more intermediate concepts to yield the set of concepts related to the target concept. The method may further include generating a multiple-choice question in which the target concept corresponds to a correct answer choice and the set of concepts related to the target concept correspond to distractors.

Abstract: Techniques for speech processing using a deep neural network (DNN) based acoustic model front-end are described. A new modeling approach directly models multi-channel audio data received from a microphone array using a first model (e.g., multi-channel DNN) that takes in raw signals and produces a first feature vector that may be used similarly to beamformed features generated by an acoustic beamformer. A second model (e.g., feature extraction DNN) processes the first feature vector and transforms it to a second feature vector having a lower dimensional representation. A third model (e.g., classification DNN) processes the second feature vector to perform acoustic unit classification and generate text data. These three models may be jointly optimized for speech processing (as opposed to individually optimized for signal enhancement), enabling improved performance despite a reduction in microphones and a reduction in bandwidth consumption during real-time processing.

Abstract: Video analysis tool systems and methods are described for a streamlined presentation feedback on a centralized platform to reduce or eliminate a delay time associated with presentation feedback. The video analysis software application tool may be configured to record a presentation to generate a video recording, analyze the video recording of the presentation based on feedback parameters and an associated neural network model, and automatically provide feedback based on the analysis of the video recording.

Abstract: In general, the subject matter described in this specification can be embodied in methods, systems, and program products for receiving s voice query at a mobile computing device and generating data that represents content of the voice query. The data is provided to a server system. A textual query that has been determined by a speech recognizer at the server system to be a textual form of at least part of the data is received at the mobile computing device. The textual query is determined to include a carrier phrase of one or more words that is reserved by a first third-party application program installed on the computing device. The first third-party application is selected, from a group of one or more third-party applications, to receive all or a part of the textual query. All or a part of the textual query is provided to the selected first application program.

Abstract: A virtual agent that utilizes an in-house natural language understanding (NLU) service and integrates a third party NLU service. The third-party NLU service is integrated with the virtual agent via a transformation script that establishes a transformation boundary through which communications are directed for adjustment and conditioning. The third party NLU service communicates with the virtual agent via an application programming interface (API). The virtual agent receives an utterance from a user via a chat session and provides the utterance to the third party NLU service. The third party NLU service may return intents, entities, and confidence, generate and return a response, and/or take actions within the cloud-based platform via the API, dependent upon the degree of integration. The virtual agent then provides a response to the user via the chat session.

Abstract: Learning means 701 learns a model for identifying an object indicated by data by using training data. First identification means 702 identifies the object indicated by the data by using the model learned by the learning means 701. Second identification means 703 identifies the object indicated by the data as an identification target used by the first identification means 702 by using a model different from the model learned by the learning means 701. The learning means 701 re-learns the model by using the training data including the label for the data determined based on the identification result derived by the second identification means 703 and the data.

Abstract: Techniques herein relate to improving quality of classification models for differentiating different user intents by improving the quality of training samples used to train the classification models. Pairs of user intents that are difficult to differentiate by classification models trained using the given training samples are identified based upon distinguishability scores (e.g., F-scores). For each of the identified pairs of intents, pairs of training samples each including a training sample associated with a first intent and a training sample associated with a second intent in the pair of intents are ranked based upon a similarity score between the two training samples in each pair of training samples. A particular pair of training samples with a highest similarity score is selected and provided as output with a suggestion for modifying the particular pair of training samples.

Abstract: A pipeline is provided for management of a pool of chat message templates for an automated dialog system. The pool of chat messages may be managed using machine learning-based clustering and feedback-based modifications. A set of chat messages may be analyzed using a machine learning model to generate different clusters of messages that are semantically related. Representative messages may be selected from each cluster and used in chat sessions according to the semantic context of the chat sessions. Based on feedback obtained during the chat sessions, metrics generated based on the feedback, and/or other data, modifications may be made to the clusters and/or the representative messages to improve the performance of the automated dialog system.

Abstract: Methods and systems for training and/or using a language selection model for use in determining a particular language of a spoken utterance captured in audio data. Features of the audio data can be processed using the trained language selection model to generate a predicted probability for each of N different languages, and a particular language selected based on the generated probabilities. Speech recognition results for the particular language can be utilized responsive to selecting the particular language of the spoken utterance. Many implementations are directed to training the language selection model utilizing tuple losses in lieu of traditional cross-entropy losses. Training the language selection model utilizing the tuple losses can result in more efficient training and/or can result in a more accurate and/or robust model—thereby mitigating erroneous language selections for spoken utterances.

Abstract: A translation method includes: selecting a source word from a source sentence; generating mapping information including location information of the selected source word mapped to the selected source word in the source sentence; and correcting a target word, which is generated by translating the source sentence, based on location information of a feature value of the target word and the mapping information.