Abstract: The method comprises receive first audio comprising speech from a user of a computing device, detecting an end of speech in the first audio, generating an ASR result based, at least in part, on a portion of the first audio prior to the detected end of speech, determining whether a valid action can be performed by a speech-enabled application installed on the computing device using the ASR result, and processing second audio when it is determined that a valid action cannot be performed by the speech-enabled application using the ASR result.

Abstract: According to some aspects, a method for aligning a first data source and a second data source during a plurality of iterations comprising a current iteration and a previous iteration is provided. The method comprises generating at least one property alignment hypothesis between at least one first property of the first data source and at least one second property of the second data source; generating a plurality of instance alignment hypotheses between a respective first plurality of instances of the first data source and a respective second plurality of instances of the second data source; and verifying at least one property alignment hypothesis and/or at least one of the plurality of instance alignment hypotheses. Generating the at least one property alignment hypothesis and/or generating the plurality of instance alignment hypotheses is based, at least in part, on at least one property alignment hypothesis and/or at least one instance alignment hypothesis verified during the previous iteration.

Abstract: Methods and apparatus for natural language understanding (NLU) processing based on user-specified interests. Information specifying a weight for each of a plurality of domains is received via a user interface. The plurality of domains each relates to a potential area of interest for the user, and the weight for a domain from among the plurality of domains indicates a level of interest for the user in the domain. A ranking classifier used to rank NLU hypotheses generated by an NLU engine is trained using training data from which features are, at least in part, based on the information specifying a weight for each of the plurality of domains.

Abstract: A method of verifying a user identity using a Web-based multimodal interface can include sending, to a remote computing device, a multimodal markup language document that, when rendered by the remote computing device, queries a user for a user identifier and causes audio of the user's voice to be sent to a multimodal, Web-based application. The user identifier and the audio can be received at about a same time from the client device. The audio can be compared with a voice print associated with the user identifier. The user at the remote computing device can be selectively granted access to the system according to a result obtained from the comparing step.

Abstract: A system, method and computer-readable storage device provides an improved speech processing approach in which hyper parameters used for speech recognition are modified dynamically or in batch mode rather than fixed statically. The method includes estimating, via a model trained on audio data and/or metadata, a set of parameters useful for performing automatic speech recognition, receiving speech at an automatic speech recognition system, applying, by the automatic speech recognition system, the set of parameters to processing the speech to yield text and outputting the text from the automatic speech recognition system.

Abstract: An ontology stores information about a domain of an automatic speech recognition (ASR) application program. The ontology is augmented with information that enables subsequent automatic generation of a speech understanding grammar for use by the ASR application program. The information includes hints about how a human might talk about objects in the domain, such as preludes (phrases that introduce an identification of the object) and postludes (phrases that follow an identification of the object).

Abstract: A method, computer program product, and computing system for obtaining encounter information of a patient encounter, wherein the encounter information includes machine vision encounter information; processing the machine vision encounter information to identify one or more humanoid shapes; and steering one or more audio recording beams toward the one or more humanoid shapes to capture audio encounter information.

Abstract: Methods described herein provide functionality for automatic speech recognition (ASR). One such embodiment performs speech recognition using received speech recognition result candidates, where the received candidates were generated by performing Statistical Language Model (SLM) based speech recognition on one or more frames of audio data. In turn, such an embodiment transmits results of the speech recognition, performed using the received speech recognition result candidates, to a user device via a communications network. Results of the speech recognition are available with lower latency than pure cloud based ASR solution.

Abstract: A method of providing a task assistant to provide an interface to an application, the method comprising activating the task assistant, the activation having an associated visual display. The method in one embodiment includes receiving input from a user through multimodal input including a plurality of speech input, typing input, and touch input, interpreting the input, and providing a formatted query to the application, receiving data from the application in response to the query, and providing a response to the user through multimodal output including a plurality of: speech output, text output, non-speech audio output, haptic output, and visual non-text output, wherein the task assistant has a plurality of active states, each of the active states having an associated visual display.

Abstract: Cascaded models may be applied to extract facts from a medical text. A first model may be applied to at least a portion of the medical text. The first model extracts at least one first medical fact. The at least one first medical fact is linked to at least first text in the at least a portion of the medical text. A second model may be applied to the first text. The second model extracts at least one second fact that is an attribute of the at least one first medical fact.

Abstract: A multi-mode voice controlled user interface is described. The user interface is adapted to conduct a speech dialog with one or more possible speakers and includes a broad listening mode which accepts speech inputs from the possible speakers without spatial filtering, and a selective listening mode which limits speech inputs to a specific speaker using spatial filtering. The user interface switches listening modes in response to one or more switching cues.

Abstract: Aspects of the disclosure are directed to natural language processing. An input interface of a computing device receives input (e.g., speech input) and generates a digital signal corresponding to that input. Text corresponding to the digital signal is obtained, and the text is processed using each of a context-free and a context-specific linguistic model to generate linguistic processing results for that text. The text and linguistic processing results may be processed using a NLU model to generate an NLU recognition result corresponding to the input received at the input interface. The text and the linguistic processing results may also be annotated and used to train a NLU model. The linguistic processing results may relate to, e.g., the tokenization of portions of the text, the normalization of portions of the text, sequences of normalizations for portions of the text, and rankings and prioritization of the linguistic processing results.

Abstract: Aspects of the disclosure are directed to natural language processing or natural language understanding and may include a determination of a probabilistic or probability-based ranking of potential results. For example, natural language input may be received such as speech or text. Natural language processing may be performed to determine one or more potential results for the input. A pairwise classifier may be used to determine a score for element pairs in the potential results. Based on the scores, probabilities for the element pairs may be determined. Based on the probabilities for the element pairs, further probabilities may be determined such as by estimating the probability that a current result is the top rank or best choice. Based on the estimated probabilities that the current result is the top rank or best choice, a ranking may be determined, which may form the basis for natural language understanding output.

Abstract: A method in accordance with the present disclosure may include receiving a message at a mobile computing device and performing natural language processing (NLP) based interpretation of the message. Embodiments may further include suggesting at least one of an action and an application configured to perform the action, the suggestion based upon, at least in part, the NLP-based interpretation of the message.

Abstract: Techniques for question answering involve receiving, from a user, a text input expressing a question in natural language. In response to the question, a text output expressing an answer to the question may be generated. A plurality of documents comprising natural language text may be analyzed, involving mapping the question to one or more hypotheses, analyzing at least one passage of text in at least one of the documents to determine whether the passage entails at least one of the hypotheses, and in response to determining that the passage entails at least one of the hypotheses, identifying the passage as providing supporting evidence for the answer to the question. The answer and the at least one passage identified as providing supporting evidence for the answer may be presented to the user in response to the text input.

Abstract: A method of providing a task assistant to provide an interface to an application is described. The method comprises receiving input from a user through multimodal input including a plurality of speech input, typing input, and touch input, interpreting the input, and providing a formatted query to the application, receiving data from the application in response to the query, and providing a response to the user through multimodal output including a plurality of: speech output, text output, non-speech audio output, haptic output, and visual non-text output.

Abstract: A system, method and computer-readable storage device are disclosed for managing a mute and unmute feature on a device which is used to communicate data in a communication conference. The method includes detecting, when the device is set to mute, whether the user is speaking and whether the speech is meant for the conference. Background noises are distinguished from the speech of the user. If the user is speaking and the device is set to mute, the device will automatically switch to and unmute setting such that people in the indication conference can hear the user speak. Facial recognition, and gaze detection or other data can also be used to determine when to automatically mute or unmute the device and can aid in inferring an intent of the user to speak to the conference participants.

Abstract: In an embodiment, a method includes determining, based on a received query and contextual information, candidate reasoners to respond to a received query to select a candidate reasoner. A reasoner or candidate reasoner is a module that translates information from a sensor, user settings, or other source, into additional or revised fields for a query. The method further includes generating, at each candidate reasoner determined, additional or revised query fields based on the contextual information and a rule of a rule database. The method further includes merging the additional query fields for each candidate reasoner based on a confidence score or other metric of each corresponding candidate reasoner. The confidence score can be based on applicability of the contextual information to the received query. The method further includes providing an enhanced query having the additional or revised query fields.

Abstract: An arrangement is described for conducting natural language dialogs with a user on a mobile device using automatic speech recognition (ASR) and multiple different dialog applications. A user interface provides for user interaction with the dialogue applications in natural language dialogs. An ASR engine processes unknown speech inputs from the user to produce corresponding speech recognition results. A dialog concept module develops dialog concept items from the speech recognition results and stores the dialog concept items and additional dialog information in a dialog concept database. A dialog processor accesses dialog concept database information and coordinates operation of the ASR engine and the dialog applications to conduct with the user a plurality of separate parallel natural language dialogs in the dialog applications.

Abstract: Techniques for training a natural language understanding (NLU) engine may include generating a first annotation of free-form text documenting a healthcare patient encounter and a link between the first annotation and a corresponding portion of the text, using the NLU engine. A second annotation of the text and a link between the second annotation and a corresponding portion of the text may be received from a human user. The first annotation and its corresponding link may be merged with the second annotation and its corresponding link. Training data may be provided to the engine in the form of the text and the merged annotations and links.