Abstract: A computer-implemented method includes receiving, by a computing device, a particular textual description of a scene. The method also includes applying a neural network for text-to-image generation to generate an output image rendition of the scene, the neural network having been trained to cause two image renditions associated with a same textual description to attract each other and two image renditions associated with different textual descriptions to repel each other based on mutual information between a plurality of corresponding pairs, wherein the plurality of corresponding pairs comprise an image-to-image pair and a text-to-image pair. The method further includes predicting the output image rendition of the scene.

Abstract: A system for predicting and summarizing medical events from electronic health records includes a computer memory storing aggregated electronic health records from a multitude of patients of diverse age, health conditions, and demographics including medications, laboratory values, diagnoses, vital signs, and medical notes. The aggregated electronic health records are converted into a single standardized data structure format and ordered arrangement per patient, e.g., into a chronological order. A computer (or computer system) executes one or more deep learning models trained on the aggregated health records to predict one or more future clinical events and summarize pertinent past medical events related to the predicted events on an input electronic health record of a patient having the standardized data structure format and ordered into a chronological order.

Abstract: A computer-implemented method includes receiving, by a computing device, a particular textual description of a scene. The method also includes applying a neural network for text-to-image generation to generate an output image rendition of the scene, the neural network having been trained to cause two image renditions associated with a same textual description to attract each other and two image renditions associated with different textual descriptions to repel each other based on mutual information between a plurality of corresponding pairs, wherein the plurality of corresponding pairs comprise an image-to-image pair and a text-to-image pair. The method further includes predicting the output image rendition of the scene.

Abstract: A method and workstation for generating a transcript of a conversation between a patient and a healthcare practitioner is disclosed. A workstation is provided with a tool for rendering of an audio recording of the conversation and generating a display of a transcript of the audio recording using a speech-to-text engine, thereby enabling inspection of the accuracy of conversion of speech to text. A tool is provided for scrolling through the transcript and rendering the portion of the audio according to the position of the scrolling. There is a highlighting in the transcript of words or phrases spoken by the patient relating to symptoms, medications or other medically relevant concepts. Additionally, there is provided a set of transcript supplement tools enabling editing of specific portions of the transcript based on the content of the corresponding portion of audio recording.

Abstract: A method and workstation for generating a transcript of a conversation between a patient and a healthcare practitioner is disclosed. A workstation is provided with a tool for rendering of an audio recording of the conversation and generating a display of a transcript of the audio recording using a speech-to-text engine, thereby enabling inspection of the accuracy of conversion of speech to text. A tool is provided for scrolling through the transcript and rendering the portion of the audio according to the position of the scrolling. There is a highlighting in the transcript of words or phrases spoken by the patient relating to symptoms, medications or other medically relevant concepts. Additionally, there is provided a set of transcript supplement tools enabling editing of specific portions of the transcript based on the content of the corresponding portion of audio recording.

Abstract: A method and system is provided for assisting a user to assign a label to words or spans of text in a transcript of a conversation between a patient and a medical professional and form groupings of such labelled words or spans of text in the transcript. The transcript is displayed on an interface of a workstation. A tool is provided for highlighting spans of text in the transcript consisting of one or more words. Another tool is provided for assigning a label to the highlighted spans of text. This tool includes a feature enabling searching through a set of predefined labels available for assignment to the highlighted span of text. The predefined labels encode medical entities and attributes of the medical entities. The interface further includes a tool for creating groupings of related highlighted spans of texts. The tools can consist of mouse action or keystrokes or a combination thereof.

Abstract: A method and workstation for generating a transcript of a conversation between a patient and a healthcare practitioner is disclosed. A workstation is provided with a tool for rendering of an audio recording of the conversation and generating a display of a transcript of the audio recording using a speech-to-text engine, thereby enabling inspection of the accuracy of conversion of speech to text. A tool is provided for scrolling through the transcript and rendering the portion of the audio according to the position of the scrolling. There is a highlighting in the transcript of words or phrases spoken by the patient relating to symptoms, medications or other medically relevant concepts. Additionally, there is provided a set of transcript supplement tools enabling editing of specific portions of the transcript based on the content of the corresponding portion of audio recording.

Abstract: A method and system is provided for assisting a user to assign a label to words or spans of text in a transcript of a conversation between a patient and a medical professional and form groupings of such labelled words or spans of text in the transcript. The transcript is displayed on an interface of a workstation. A tool is provided for highlighting spans of text in the transcript consisting of one or more words. Another tool is provided for assigning a label to the highlighted spans of text. This tool includes a feature enabling searching through a set of predefined labels available for assignment to the highlighted span of text. The predefined labels encode medical entities and attributes of the medical entities. The interface further includes a tool for creating groupings of related highlighted spans of texts. The tools can consist of mouse action or key strokes or a combination thereof.

Abstract: A method and workstation for generating a transcript of a conversation between a patient and a healthcare practitioner is disclosed. A workstation is provided with a tool for rendering of an audio recording of the conversation and generating a display of a transcript of the audio recording using a speech-to-text engine, thereby enabling inspection of the accuracy of conversion of speech to text. A tool is provided for scrolling through the transcript and rendering the portion of the audio according to the position of the scrolling. There is a highlighting in the transcript of words or phrases spoken by the patient relating to symptoms, medications or other medically relevant concepts. Additionally, there is provided a set of transcript supplement tools enabling editing of specific portions of the transcript based on the content of the corresponding portion of audio recording.

Abstract: A system for predicting and summarizing medical events from electronic health records includes a computer memory storing aggregated electronic health records from a multitude of patients of diverse age, health conditions, and demographics including medications, laboratory values, diagnoses, vital signs, and medical notes. The aggregated electronic health records are converted into a single standardized data structure format and ordered arrangement per patient, e.g., into a chronological order. A computer (or computer system) executes one or more deep learning models trained on the aggregated health records to predict one or more future clinical events and summarize pertinent past medical events related to the predicted events on an input electronic health record of a patient having the standardized data structure format and ordered into a chronological order.

Abstract: A computer-implemented method includes receiving, by a computing device, a particular textual description of a scene. The method also includes applying a neural network for text-to-image generation to generate an output image rendition of the scene, the neural network having been trained to cause two image renditions associated with a same textual description to attract each other and two image renditions associated with different textual descriptions to repel each other based on mutual information between a plurality of corresponding pairs, wherein the plurality of corresponding pairs comprise an image-to-image pair and a text-to-image pair. The method further includes predicting the output image rendition of the scene.

Abstract: A method for automatically generating a note summarizing a conversation between a patient and a healthcare provider is disclosed. A workstation is provided with a tool for rendering an audio recording of the conversation and a display for displaying a transcript of the audio recording obtained from a speech-to-text engine. The display of the workstation includes first transcript region for display of the transcript and a second note region for simultaneous displaying of elements of a note summarizing the conversation. Words or phrases in the transcript related to medical topics relating to the patient are extracted with the aid of a trained machine learning model. The extracted words or phrases are highlighted in the transcript and displayed in the note region.

Abstract: The present disclosure provides systems and methods that generating health diagnostic information from an audio recording. A computing system can include a machine-learned health model comprising that includes a sound model trained to receive data descriptive of a patient audio recording and output sound description data. The computing system can include a diagnostic model trained to receive the sound description data and output a diagnostic score. The computing system can include at least one tangible, non-transitory computer-readable medium that stores instructions that, when executed, cause the processor to perform operations. The operations can include obtaining the patient audio recording; inputting data descriptive of the patient audio recording into the sound model; receiving, as an output of the sound model, the sound description data; inputting the sound description data into the diagnostic model; and receiving, as an output of the diagnostic model, the diagnostic score.

Abstract: A method is described for training a predictive model which increases the interpretability and trustworthiness of the model for end-users. The model is trained from data having multitude of features. Each feature is associated with a real value and a time component. Many predicates (atomic elements for training the model) are defined as binary functions operating on the features, and typically time sequences of the features or logical combinations thereof. The predicates can be limited to those functions which have human understandability or encode expert knowledge relative to a predication task of the model. We iteratively train a boosting model with input from an operator or human-in-the-loop. The human-in-the-loop is provided with tools to inspect the model as it is iteratively built and remove one or more of the predicates in the model, e.g. if it does not have indicia of trustworthiness, is not causally related to a prediction of the model, or is not understandable.

Abstract: A method and workstation for generating a transcript of a conversation between a patient and a healthcare practitioner is disclosed. A workstation is provided with a tool for rendering of an audio recording of the conversation and generating a display of a transcript of the audio recording using a speech-to-text engine, thereby enabling inspection of the accuracy of conversion of speech to text. A tool is provided for scrolling through the transcript and rendering the portion of the audio according to the position of the scrolling. There is a highlighting in the transcript of words or phrases spoken by the patient relating to symptoms, medications or other medically relevant concepts. Additionally, there is provided a set of transcript supplement tools enabling editing of specific portions of the transcript based on the content of the corresponding portion of audio recording.

Abstract: A method and workstation for generating a transcript of a conversation between a patient and a healthcare practitioner is disclosed. A workstation is provided with a tool for rendering of an audio recording of the conversation and generating a display of a transcript of the audio recording using a speech-to-text engine, thereby enabling inspection of the accuracy of conversion of speech to text. A tool is provided for scrolling through the transcript and rendering the portion of the audio according to the position of the scrolling. There is a highlighting in the transcript of words or phrases spoken by the patient relating to symptoms, medications or other medically relevant concepts. Additionally, there is provided a set of transcript supplement tools enabling editing of specific portions of the transcript based on the content of the corresponding portion of audio recording.

Abstract: A machine learning method is described for generating labels for members of a training set where the labels are not directly available in the training set data. In a first stage of the method an iterative process is used to gradually build up a list of features (“partition features” herein) which are conceptually related to the class label using a human-in-the loop (expert). In a second part of the process we generate labels for the members of the training set, build up a boosting model using the labeling to come up with additional partition features, score the labeling of the training set members from the boosting model, and then with the human-in-the-loop evaluate a labels assigned to a small subset of the members depending on their score. The labels assigned to some or all of those members in the subset may be flipped depending on the evaluation. The final outcome of the process is an interpretable model that explains how the labels were generated and a labeled set of training data.

Abstract: A method and workstation for generating a transcript of a conversation between a patient and a healthcare practitioner is disclosed. A workstation is provided with a tool for rendering of an audio recording of the conversation and generating a display of a transcript of the audio recording using a speech-to-text engine, thereby enabling inspection of the accuracy of conversion of speech to text. A tool is provided for scrolling through the transcript and rendering the portion of the audio according to the position of the scrolling. There is a highlighting in the transcript of words or phrases spoken by the patient relating to symptoms, medications or other medically relevant concepts. Additionally, there is provided a set of transcript supplement tools enabling editing of specific portions of the transcript based on the content of the corresponding portion of audio recording.

Abstract: A method and workstation for generating a transcript of a conversation between a patient and a healthcare practitioner is disclosed. A workstation is provided with a tool for rendering of an audio recording of the conversation and generating a display of a transcript of the audio recording using a speech-to-text engine, thereby enabling inspection of the accuracy of conversion of speech to text. A tool is provided for scrolling through the transcript and rendering the portion of the audio according to the position of the scrolling. There is a highlighting in the transcript of words or phrases spoken by the patient relating to symptoms, medications or other medically relevant concepts. Additionally, there is provided a set of transcript supplement tools enabling editing of specific portions of the transcript based on the content of the corresponding portion of audio recording.

Abstract: A method and system is provided for assisting a user to assign a label to words or spans of text in a transcript of a conversation between a patient and a medical professional and form groupings of such labelled words or spans of text in the transcript. The transcript is displayed on an interface of a workstation. A tool is provided for highlighting spans of text in the transcript consisting of one or more words. Another tool is provided for assigning a label to the highlighted spans of text. This tool includes a feature enabling searching through a set of predefined labels available for assignment to the highlighted span of text. The predefined labels encode medical entities and attributes of the medical entities. The interface further includes a tool for creating groupings of related highlighted spans of texts. The tools can consist of mouse action or key strokes or a combination thereof.