Abstract: An apparatus for analyzing multimodal data includes an image processor configured to generate an activation embedding vector based on an index of an activation map obtained from image data through a convolutional neutral network, a text processor configured to receive text data to generate a text embedding vector, a vector concatenator configured to concatenate the activation embedding vector and the text embedding vector to each other to generate a concatenated embedding vector, and an encoder configured to generate a multimodal representation vector in consideration of an influence between elements constituting the concatenated embedding vector based on self-attention.

Abstract: A method for image analysis according to an embodiment may include generating a prediction result for an original image using a pre-trained image classification model, learning a plurality of masks using the original image, the prediction result, and the image classification model, and generating a map visualizing a importance of each area of the original image for the prediction result based on at least one of the plurality of masks.

Abstract: A method for image analysis according to an embodiment may include generating a prediction result for an original image using a pre-trained image classification model, learning a plurality of masks using the original image, the prediction result, and the image classification model, and generating a map visualizing a importance of each area of the original image for the prediction result based on at least one of the plurality of masks.