Abstract: A system (100) is extracting targeted medical information from clinical notes stored in memory (120). The system (100) includes a preprocessing module (120a) configured to retrieve from the memory (120) a sequence of clinical texts of electronic health records, and to tokenize the sequence of clinical texts to obtain a sequence of input tokens. The system (100) further includes a sequence to structure model module (120b) configured to transform, using a trained natural language based transformer, the sequence of input tokens into a sequence of structured output tokens. The system (100) further includes a post-processing unit (110) configured to obtain annotated text-label pairs of the clinical texts from the structure output tokens.

Abstract: A system (1000) for automated question answering, including: semantic space (210) generated from a corpus of questions and answers; a user interface (1030) configured to receive a question; and a processor (1100) comprising: (i) a question decomposition engine (1050) configured to decompose the question into a domain, a keyword, and a focus word; (ii) a question similarity generator (1060) configured to identify one or more questions in a semantic space using the decomposed question; (iii) an answer extraction and ranking engine (1080) configured to: extract, from the semantic space, answers associated with the one or more identified questions; and identify one or more of the extracted answers as a best answer; and (iv) an answer tuning engine (1090) configured to fine-tune the identified best answer using one or more of the domain, keyword, and focus word; wherein the fine-tuned answer is provided to the user via the user interface.

Abstract: Techniques are described herein for training and applying memory neural networks, such as “condensed” memory neural networks (“C-MemNN”) and/or “average” memory neural networks (“A-MemNN”). In various embodiments, the memory neural networks may be iteratively trained using training data in the form of free form clinical notes and clinical reference documents. In various embodiments, during each iteration of the training, a so-called “condensed” memory state may be generated and used as part of the next iteration. Once trained, a free form clinical note associated with a patient may be applied as input across the memory neural network to predict one or more diagnoses or outcomes of the patient.

Abstract: The described embodiments relate to systems, methods, and apparatus for providing a multimodal deep memory network (200) capable of generating patient diagnoses (222). The multimodal deep memory network can employ different neural networks, such as a recurrent neural network and a convolution neural network, for creating embeddings (204, 214, 216) from medical images (212) and electronic health records (206). Connections between the input embeddings (204) and diagnoses embeddings (222) can be based on an amount of attention that was given to the images and electronic health records when creating a particular diagnosis. For instance, the amount of attention can be characterized by data (110) that is generated based on sensors that monitor eye movements of clinicians observing the medical images and electronic health records.

Abstract: Techniques are described herein for training and applying memory neural networks, such as “condensed” memory neural networks (“C-MemNN”) and/or “average” memory neural networks (“A-MemNN”). In various embodiments, the memory neural networks may be iteratively trained using training data in the form of free form clinical notes and clinical reference documents. In various embodiments, during each iteration of the training, a so-called “condensed” memory state may be generated and used as part of the next iteration. Once trained, a free form clinical note associated with a patient may be applied as input across the memory neural network to predict one or more diagnoses or outcomes of the patient.

Abstract: A system and method for visualizing and annotating temporal trends of an abnormal condition in patient data. A classification and visualization module detects one or more conditions in one or more images, e.g. X-ray images, and visualizes the condition on the image. A temporal disease state extraction module analyzes text, e.g. radiology reports, for indications of a change in the condition. A multimodal disease state comparison module fuses the extracted data into a compact representation of the condition changes over time.

Abstract: A system and method for unsupervised training of a text report identification machine learning model, including: labeling a first set of unlabeled text reports using a seed dictionary to identify concepts in the unlabeled text reports; inputting images associated with the first set of seed-labeled text reports into an auto-encoder to obtain an encoded first set of images; calculating a set of first correlation matrices as a dot product of the first encoded images with their associated text report features; determining a distance between the set of first correlation matrices and a filter bank value associated with the auto-encoder; identifying a first set of validated images as the images in the first set of images that have a distance less than a threshold value; and training the text report machine learning model using the labeled text reports associated with the set of first validated images.

Abstract: A system and method for training a text report identification machine learning model and an image identification machine learning model, including: initially training a text report machine learning model, using a labeled set of text reports including text pre-processing the text report and extracting features from the pre-processed text report, wherein the extracted features are input into the text report machine learning model; initially training an image machine learning model, using a labeled set of images; applying the initially trained text report machine learning model to a first set of unlabeled text reports with associated images to label the associated images; selecting a first portion of labeled associated images; re-training the image machine learning model using the selected first portion of labeled associated images; applying the initially trained image machine learning model to a first set of unlabeled images with associated text reports to label the associated text reports; selecting a first po

Abstract: A medical information retrieval system comprises a natural language processing system that processes a vocal user query to identify key words and phrases. These key words and phrases are provided to an inferencing engine that provides a set of knowledge-based inferences from medical knowledge sources, based on these key words and phrases. Thereafter, these knowledge-based inferences are provided to an information retrieval engine that retrieves a corresponding plurality of medical articles based on these knowledge-based inferences, and ranks each with respect to the knowledge-based inferences. A summary engine receives the ranked articles and creates a model based on the topical keywords and candidate sentences found in the highly ranked articles. A paraphrase engine processes the candidate sentences to provide a summary response based on a knowledge-based paraphrase model. An audio output device renders the summary report as the response to the user's original vocal query.

Abstract: Methods and systems for generating a question from free text. The system is trained on a corpus of data and receives a tuple consisting of a paragraph (free text), a focused fact, and a question type. The system implements a language model to find the most optimal combination of words to return a question for the paragraph about the focused fact.

Abstract: A method for presenting do-it-yourself (DIY) videos to a user related to a user task by a DIY video system, including: receiving a user query including a first image file and a text question from a user regarding the current state of the user task; extracting entities from the first image file to create entity data; extracting question information from the text question; extracting from a DIY video index a video segment related to the user task based upon the entity data and the question information; and presenting the extracted video segment to the user.

Abstract: Methods and systems for generating a question from free text. The system is trained on a corpus of data and receives a tuple consisting of a paragraph (free text), a focused fact, and a question type. The system implements a language model to find the most optimal combination of words to return a question for the paragraph about the focused fact.

Abstract: A device, system, and method classifies a cognitive bias in a microblog relative to healthcare-centric evidence. The method performed at a microblog server includes receiving a selection from a clinician, the selection indicating a health-related topic. The method includes determining evidence data of the health-related topic from validated information sources. The method includes receiving a microblog, the microblog associated with the health-related topic. The method includes determining a cognitive bias of the microblog based on the evidence data.

Abstract: Methods and systems for interacting with a user. Systems in accordance with various embodiments described herein provide a collection of models that are each trained to perform a specific function. These models may be categorized into static models that are trained on an existing corpus of information and dynamic models that are trained based on real-time interactions with users. Collectively, the models provide appropriate communications for a user.

Abstract: A system (1000) for automated question answering, including: semantic space (210) generated from a corpus of questions and answers; a user interface (1030) configured to receive a question; and a processor (1100) comprising: (i) a question decomposition engine (1050) configured to decompose the question into a domain, a keyword, and a focus word; (ii) a question similarity generator (1060) configured to identify one or more questions in a semantic space using the decomposed question; (iii) an answer extraction and ranking engine (1080) configured to: extract, from the semantic space, answers associated with the one or more identified questions; and identify one or more of the extracted answers as a best answer; and (iv) an answer tuning engine (1090) configured to fine-tune the identified best answer using one or more of the domain, keyword, and focus word; wherein the fine-tuned answer is provided to the user via the user interface.

Abstract: A system (500) for automated clinical diagnosis includes: a knowledge graph (310, 510) generated using a curated corpus of medical information (520) and comprising a plurality of nodes; a user interface (512) configured to receive input comprising information about at least one patient symptom (316) and at least one patient demographic parameter (318); and a processor (530) configured to extract the at least one patient symptom and demographic parameter, and further configured to: (i) weight the extracted patient symptom; (ii) query the knowledge graph to generate a diagnosis graph as a subset of the knowledge graph; (iii) identify a ranked list of medical conditions for the patient from the diagnosis graph; and (iv) adjust, based on the extracted at least one demographic parameter about the patient, the ranking of the ranked list; wherein the identified medical conditions are provided to the user via the user interface.

Abstract: A medical information retrieval system comprises a natural language processing system that processes a vocal user query to identify key words and phrases. These key words and phrases are provided to an inferencing engine that provides a set of knowledge-based inferences from medical knowledge sources, based on these key words and phrases. Thereafter, these knowledge-based inferences are provided to an information retrieval engine that retrieves a corresponding plurality of medical articles based on these knowledge-based inferences, and ranks each with respect to the knowledge-based inferences. A summary engine receives the ranked articles and creates a model based on the topical keywords and candidate sentences found in the highly ranked articles. A paraphrase engine processes the candidate sentences to provide a summary response based on a knowledge-based paraphrase model. An audio output device renders the summary report as the response to the user's original vocal query.

Abstract: The described embodiments relate to systems, methods, and apparatus for providing a multimodal deep memory network (200) capable of generating patient diagnoses (222). The multimodal deep memory network can employ different neural networks, such as a recurrent neural network and a convolution neural network, for creating embeddings (204, 214, 216) from medical images (212) and electronic health records (206). Connections between the input embeddings (204) and diagnoses embeddings (222) can be based on an amount of attention that was given to the images and electronic health records when creating a particular diagnosis. For instance, the amount of attention can be characterized by data (110) that is generated based on sensors that monitor eye movements of clinicians observing the medical images and electronic health records.

Abstract: In adverse drug event (ADE) monitoring and reporting, drug-related messages (60) are detected in one or more social media message streams as messages that include a name of a monitored drug. ADE reports (62) are extracted from the drug-related messages using an ADE classifier (46). The extracted ADE reports are validated by comparison with known ADEs of the monitored drug stored in an ADE knowledge base (64). Extracted ADE reports that fail the validating are collected in a non-validated ADE reports database (72). A report (74) is generated including information on at least one previously unrecognized ADE for which extracted ADE reports in the non-validated ADE reports database satisfy a previously unrecognized ADE criterion (in terms of number of messages or number of unique patients reporting the ADE).

Abstract: Techniques are described herein for training and applying memory neural networks, such as “condensed” memory neural networks (“C-MemNN”) and/or “average” memory neural networks (“A-MemNN”). In various embodiments, the memory neural networks may be iteratively trained using training data in the form of free form clinical notes and clinical reference documents. In various embodiments, during each iteration of the training, a so-called “condensed” memory state may be generated and used as part of the next iteration. Once trained, a free form clinical note associated with a patient may be applied as input across the memory neural network to predict one or more diagnoses or outcomes of the patient.