Abstract: A method (100) for generating a textual description of a medical image, comprising: receiving (130) a medical image of an anatomical region, the image comprising one or more abnormalities; segmenting (140) the anatomical region in the received medical image from a remainder of the image; identifying (150) at least one of the one or more abnormalities in the segmented anatomical region; extracting (160) one or more features from the identified abnormality; generating (170), using the extracted features and a trained text generation model, a textual description of the identified abnormality; and reporting (180), via a user interface of the system, the generated textual description of the identified abnormality.

Abstract: Techniques are described for training and/or utilizing sub-agent machine learning models to generate candidate dialog responses. In various implementations, a user-facing dialog agent (202, 302), or another component on its behalf, selects one of the candidate responses which is closest to user defined global priority objectives (318). Global priority objectives can include values (306) for a variety of dialog features such as emotion, confusion, objective-relatedness, personality, verbosity, etc. In various implementations, each machine learning model includes an encoder portion and a decoder portion. Each encoder portion and decoder portion can be a recurrent neural network (RNN) model, such as a RNN model that includes at least one memory layer, such as a long short-term memory (LSTM) layer.

Abstract: A method (100) for generating a textual description of a medical image, comprising: receiving (130) a medical image of an anatomical region, the image comprising one or more abnormalities; segmenting (140) the anatomical region in the received medical image from a remainder of the image; identifying (150) at least one of the one or more abnormalities in the segmented anatomical region; extracting (160) one or more features from the identified abnormality; generating (170), using the extracted features and a trained text generation model, a textual description of the identified abnormality; and reporting (180), via a user interface of the system, the generated textual description of the identified abnormality.

Abstract: A system (1000) for automated question answering, including: semantic space (210) generated from a corpus of questions and answers; a user interface (1030) configured to receive a question; and a processor (1100) comprising: (i) a question decomposition engine (1050) configured to decompose the question into a domain, a keyword, and a focus word; (ii) a question similarity generator (1060) configured to identify one or more questions in a semantic space using the decomposed question; (iii) an answer extraction and ranking engine (1080) configured to: extract, from the semantic space, answers associated with the one or more identified questions; and identify one or more of the extracted answers as a best answer; and (iv) an answer tuning engine (1090) configured to fine-tune the identified best answer using one or more of the domain, keyword, and focus word; wherein the fine-tuned answer is provided to the user via the user interface.

Abstract: A method is provided that includes: providing data inputs to a machine learning model, where the data inputs include electronic patient data obtained from electronic records describing a health history of the patient. The method includes receiving an output from the machine learning model, where the output is generated based on the machine learning model processing the data inputs and includes identified care gaps for the patient. The method includes determining a treatment effect for each of the identified care gaps and assigning a treatment effect score to each of the identified care gaps. The method includes prioritizing the identified care gaps based on the treatment effect score assigned thereto and, based on the prioritization, determining one or more recommended patient actions for the patient. The method includes generating and transmitting an electronic communication that describes the one or more recommended patient actions for the patient.

Abstract: A method for determining the answer to a query in a document, including: encoding, by an encoder, the query and the document; generating a query-aware context encodings G by a bidirectional attention system using the encoded query and the encoded document; performing a hierarchical self-attention on the query aware document by a hierarchical self-attention system by applying a word to word attention and a word to sentence attention mechanism resulting in a matrix M; and determining the starting word and the ending word of the answer in the document by a span detector based upon the matrix M.

Abstract: Techniques are described herein for training and applying memory neural networks, such as “condensed” memory neural networks (“C-MemNN”) and/or “average” memory neural networks (“A-MemNN”). In various embodiments, the memory neural networks may be iteratively trained using training data in the form of free form clinical notes and clinical reference documents. In various embodiments, during each iteration of the training, a so-called “condensed” memory state may be generated and used as part of the next iteration. Once trained, a free form clinical note associated with a patient may be applied as input across the memory neural network to predict one or more diagnoses or outcomes of the patient.

Abstract: Techniques are described herein for drawing conclusions using free form texts and external resources. In various embodiments, free form input data (202) may be segmented (504) into a plurality of input data segments. A first input data segment may be compared (510) with an external resource (304) to identify a first candidate conclusion. A reinforcement learning trained agent (310) may be applied (512) to make a first determination of whether to accept or reject the first candidate conclusion. Similar actions may be performed with a second input data segment to make a second determination of whether to accept or reject a second candidate conclusion. A final conclusion may be presented (522) based on the first and second determinations of the reinforcement learning trained agent with respect to at least the first candidate conclusion and the second candidate conclusion.

Abstract: The described embodiments relate to systems, methods, and apparatus for providing a multimodal deep memory network (200) capable of generating patient diagnoses (222). The multimodal deep memory network can employ different neural networks, such as a recurrent neural network and a convolution neural network, for creating embeddings (204, 214, 216) from medical images (212) and electronic health records (206). Connections between the input embeddings (204) and diagnoses embeddings (222) can be based on an amount of attention that was given to the images and electronic health records when creating a particular diagnosis. For instance, the amount of attention can be characterized by data (110) that is generated based on sensors that monitor eye movements of clinicians observing the medical images and electronic health records.

Abstract: Techniques are described herein for training and applying memory neural networks, such as “condensed” memory neural networks (“C-MemNN”) and/or “average” memory neural networks (“A-MemNN”). In various embodiments, the memory neural networks may be iteratively trained using training data in the form of free form clinical notes and clinical reference documents. In various embodiments, during each iteration of the training, a so-called “condensed” memory state may be generated and used as part of the next iteration. Once trained, a free form clinical note associated with a patient may be applied as input across the memory neural network to predict one or more diagnoses or outcomes of the patient.

Abstract: The present disclosure relates to preserving context in a conversation between a user (101) and a digital assistant device (102). During training, the digital assistant device (102) is provided with a plurality of conversations having a plurality of dialogues. Each of the plurality of dialogue is assigned an ID based on a context. Further, two or more test queries having a same context is provided as input and the two are more queries are assigned an ID based on the context. Thereafter, the digital assistant device (102) is configured to retrieve one or more dialogues from the plurality of dialogues where the ID of the one or more dialogues match the ID of the two or more queries. In real-time, one or more queries are received and based on a context of the one or more queries, one or more dialogues are retrieved and are provided to the user.

Abstract: A method for determining, from a document, an answer to a query using a query answering system, comprising: (i) encoding, using an encoder, one or more documents; (ii) encoding a received query; (iii) generating, using an attention mechanism, a query-aware document representation comprising alignment between one or more words in one of the plurality of documents and one or more words in the query; (iv) generating, using a hierarchical self-attention mechanism, a word-to-sentence alignment of the query-aware document representation; (v) labeling, using a conditional random field classifier, each of a plurality of words in the word-to-sentence alignment with one of a one of a plurality of different sequence identifiers, resulting in possible labeled answering spans; and (vi) generating, from the one or more possible labeled answering spans, a response to the query.

Abstract: Techniques described herein relate to semi-supervised training and application of stacked autoencoders and other classifiers for predictive and other purposes. In various embodiments, a semi-supervised model (108) may be trained for sentence classification, and may combine what is referred to herein as a “residual stacked de-noising autoencoder” (“RSDA”) (220), which may be unsupervised, with a supervised classifier (218) such as a classification neural network (e.g., a multilayer perceptron, or “MLP”). In various embodiments, the RSDA may be a stacked denoising autoencoder that may or may not include one or more residual connections. If present, the residual connections may help the RSDA “remember” forgotten information across multiple layers. In various embodiments, the semi-supervised model may be trained with unlabeled data (for the RSDA) and labeled data (for the classifier) simultaneously.

Abstract: Methods and systems for generating text from a haptic-based input. The system may include an interface for receiving a haptic-based input and a processor executing instructions stored on a memory and providing a model. The model is configured to at least receive the haptic-based input and supply a text describing the haptic-based input using the interface.

Abstract: Techniques are provided for generating and applying a granular attention hierarchical neural network model to classify a document. In various embodiments, data indicative of the document may be obtained (102) and processed (104) into a first layer of two or more layers of a hierarchical network model using a dual granularity attention mechanism to generate first layer output data, wherein the dual granularity attention mechanism weighs some portions of the data indicative of the document more heavily. Some portions of the data indicative of the document are integrated into the hieratical network model during training of the dual granularity attention mechanism. The first layer output data may be processed (106) in the second of two or more layers of the hierarchical network model to generate second layer output data. A classification label can be generated (108) from the second layer output data.

Abstract: Techniques are described for training and/or utilizing sub-agent machine learning models to generate candidate dialog responses. In various implementations, a user-facing dialog agent (202, 302), or another component on its behalf, selects one of the candidate responses which is closest to user defined global priority objectives (318). Global priority objectives can include values (306) for a variety of dialog features such as emotion, confusion, objective-relatedness, personality, verbosity, etc. In various implementations, each machine learning model includes an encoder portion and a decoder portion. Each encoder portion and decoder portion can be a recurrent neural network (RNN) model, such as a RNN model that includes at least one memory layer, such as a long short-term memory (LSTM) layer.

Abstract: Methods and systems for generating a question from free text. The system is trained on a corpus of data and receives a tuple consisting of a paragraph (free text), a focused fact, and a question type. The system implements a language model to find the most optimal combination of words to return a question for the paragraph about the focused fact.

Abstract: Various embodiments described herein relate to a method, system, and non-transitory machine-readable medium including one or more of the following: extracting a first concept from input data presented for processing by a downstream function; identifying external data from an external resource based on the first concept; extracting a second concept from the external data; revising the first concept based on the second concept to produce a revised concept, wherein revising includes: applying a machine learning agent to determine whether to keep the first concept or adopt the second concept, and adopting the second concept in place of the first concept for use as the revised concept based on a decision by the machine learning agent to adopt the second concept; and further processing the revised concept according to the downstream function to generate an output.

Abstract: A system, method and device for determining and notifying a clinician of information relevant to the clinician. The method that is performed by the device or system includes identifying at least one keyword in a user profile of a clinician, identifying at least one content word in a new information item, determining a relevance score between the new information item and the clinician based on the at least one keyword and the at least one content word and when the relevance score is above a predetermined threshold value, generating a notification for the clinician indicating the new information item.

Abstract: A system for automated question answering, comprising: a user interface configured to receive a query from a user; a question decomposition engine configured to decompose the query into one or more sub-questions and one or more contexts, and to align the sub-questions with contexts to generate question-context pairs; a query engine configured to query one or more answer resources with the question-context pairs to identify information likely to comprise an answer; and an answer generator configured to: (i) generate question-context-answer triples using the identified information from the query engine; (ii) select a generated question-context-answer triple comprising information most likely to comprise an answer to the identified sub-question; (iii) extract from the selected question-context-answer triple a portion of the associated information comprising an answer to the identified sub-question; and (iv) generate a natural language answer comprising a response to the query posed by the user.