Abstract: For training data pairs comprising training text (a radiological report) and training images (radiological images associated with the radiological report), a first encoder network determines word embeddings for the training text. A concept is generated from the operation of layers of the first encoder network, which is regularized by a first loss between the generated concept and a labeled concept for the training text. A second encoder network determines features for the training image. A heatmap is generated from the operation of layers of the second encoder network, which is regularized by a second loss between the generated heatmap and a labeled heatmap for the training image. A categorical cross entropy loss is calculated between a diagnostic quality category (classified by an error encoder) and a labeled diagnostic quality category for the training data pair. A total loss function comprising the first, second, and categorical cross entropy losses is minimized.

Abstract: For training data pairs comprising training text (a radiological report) and training images (radiological images associated with the radiological report), a first encoder network determines word embeddings for the training text. A concept is generated from the operation of layers of the first encoder network, which is regularized by a first loss between the generated concept and a labeled concept for the training text. A second encoder network determines features for the training image. A heatmap is generated from the operation of layers of the second encoder network, which is regularized by a second loss between the generated heatmap and a labeled heatmap for the training image. A categorical cross entropy loss is calculated between a diagnostic quality category (classified by an error encoder) and a labeled diagnostic quality category for the training data pair. A total loss function comprising the first, second, and categorical cross entropy losses is minimized.