Abstract: Techniques are described for training and/or utilizing sub-agent machine learning models to generate candidate dialog responses. In various implementations, a user-facing dialog agent (202, 302), or another component on its behalf, selects one of the candidate responses which is closest to user defined global priority objectives (318). Global priority objectives can include values (306) for a variety of dialog features such as emotion, confusion, objective-relatedness, personality, verbosity, etc. In various implementations, each machine learning model includes an encoder portion and a decoder portion. Each encoder portion and decoder portion can be a recurrent neural network (RNN) model, such as a RNN model that includes at least one memory layer, such as a long short-term memory (LSTM) layer.

Abstract: Techniques are described for training and/or utilizing sub-agent machine learning models to generate candidate dialog responses. In various implementations, a user-facing dialog agent (202, 302), or another component on its behalf, selects one of the candidate responses which is closest to user defined global priority objectives (318). Global priority objectives can include values (306) for a variety of dialog features such as emotion, confusion, objective-relatedness, personality, verbosity, etc. In various implementations, each machine learning model includes an encoder portion and a decoder portion. Each encoder portion and decoder portion can be a recurrent neural network (RNN) model, such as a RNN model that includes at least one memory layer, such as a long short-term memory (LSTM) layer.

Abstract: According to an aspect, there is provided a computer-implemented method of structuring content for training an artificial intelligence model, the method comprising: receiving (S11) input content associated with medical device documentation; converting (S12) the input content to a data interchange format; extracting (S13) a plurality of key terms from the converted input content; extracting (S14) a plurality of key phrases from the converted input content; receiving (S15) validation of the key terms and the key phrases from a supervisor; and building (S16) a dialogue, for training the artificial intelligence model, based on at least some of the validated key terms and the validated key phrases, wherein the dialogue comprises a series of statements.

Abstract: Techniques are described for training and/or utilizing sub-agent machine learning models to generate candidate dialog responses. In various implementations, a user-facing dialog agent (202, 302), or another component on its behalf, selects one of the candidate responses which is closest to user defined global priority objectives (318). Global priority objectives can include values (306) for a variety of dialog features such as emotion, confusion, objective-relatedness, personality, verbosity, etc. In various implementations, each machine learning model includes an encoder portion and a decoder portion. Each encoder portion and decoder portion can be a recurrent neural network (RNN) model, such as a RNN model that includes at least one memory layer, such as a long short-term memory (LSTM) layer.

Abstract: A system for automated question answering, comprising: a user interface configured to receive a query from a user; a question decomposition engine configured to decompose the query into one or more sub-questions and one or more contexts, and to align the sub-questions with contexts to generate question-context pairs; a query engine configured to query one or more answer resources with the question-context pairs to identify information likely to comprise an answer; and an answer generator configured to: (i) generate question-context-answer triples using the identified information from the query engine; (ii) select a generated question-context-answer triple comprising information most likely to comprise an answer to the identified sub-question; (iii) extract from the selected question-context-answer triple a portion of the associated information comprising an answer to the identified sub-question; and (iv) generate a natural language answer comprising a response to the query posed by the user.

Abstract: A non-transitory computer-readable medium stores instructions readable and executable by at least one electronic processor (20) to perform an augmented reality (AR) content generation method (100). The method includes: acquiring, with a camera (14) of an AR device (13), one or more images of a component of a medical imaging or medical therapy device (12); receiving, from a microphone (15) of the AR device, a triggering audio segment; generating one or more query data structures from both the one or more images and the triggering audio segment; retrieving AR instructional content related to the medical imaging or medical therapy device matching the generated one or more query data structures from a database (26); and outputting the AR instructional content one or more of (i) displayed superimposed on video displayed by the AR device and/or (ii) displayed on a head mounted display of the AR device and/or (iii) as audio content via a loudspeaker (27) of the AR device.

Abstract: A method for generating a textual description from a medical image, comprising: receiving a medical image having a first modality to a system configured to generate a textual description of the medical image; determining, using an imaging modality classification module, that the first modality is a specific one of a plurality of different modalities; determining, using an anatomy classification module, that the medical image comprises information about a specific portion of an anatomy; identifying, by an orchestrator module based at least on the determined first modality, which of a plurality of different text generation models to utilize to generate a textual description from the medical image; generating, by a text generation module utilizing the identified text generation model, a textual description from the medical image; and reporting, via a user interface of the system, the generated textual description.

Abstract: A system for automated question answering, comprising: a user interface configured to receive a query from a user; a question decomposition engine configured to decompose the query into one or more sub-questions and one or more contexts, and to align the sub-questions with contexts to generate question-context pairs; a query engine configured to query one or more answer resources with the question-context pairs to identify information likely to comprise an answer; and an answer generator configured to: (i) generate question-context-answer triples using the identified information from the query engine; (ii) select a generated question-context-answer triple comprising information most likely to comprise an answer to the identified sub-question; (iii) extract from the selected question-context-answer triple a portion of the associated information comprising an answer to the identified sub-question; and (iv) generate a natural language answer comprising a response to the query posed by the user.

Abstract: A method for generating a textual description from a medical image, comprising: receiving a medical image having a first modality to a system configured to generate a textual description of the medical image; determining, using an imaging modality classification module, that the first modality is a specific one of a plurality of different modalities; determining, using an anatomy classification module, that the medical image comprises information about a specific portion of an anatomy; identifying, by an orchestrator module based at least on the determined first modality, which of a plurality of different text generation models to utilize to generate a textual description from the medical image; generating, by a text generation module utilizing the identified text generation model, a textual description from the medical image; and reporting, via a user interface of the system, the generated textual description.