Abstract: The disclosure relates to method and system for determining correctness of predictions performed by deep learning model. The method includes extracting a neuron activation pattern of a layer of the deep learning model with respect to the input data, and generating an activation vector based on the extracted neuron activation pattern. The method further includes determining the correctness of the prediction performed by the deep learning model with respect to the input data using a prediction validation model and based on the activation vector. The prediction validation model is a machine learning model that has been generated and trained using training activation vectors derived from correctly predicted test dataset and incorrectly predicted test dataset of the deep learning model. The method further includes providing the correctness of the prediction performed by the deep learning model with respect to the input data for subsequent rendering or subsequent processing.

Abstract: A method and system of identifying misclassification in a neural network is disclosed. The method includes generating for an input image, an image heatmap associated with each of a plurality of layers of a neural network. The method further includes determining for the input image, an activation value at each of the plurality of layers based on the associated image heatmap. The method includes identifying for the input image, a first pattern of triggered neurons of the plurality of layers based on the activation value generated for the plurality of layers. The method further includes comparing for each of the plurality of classes, a second pattern of triggered neurons with the first pattern of triggered neurons identified at the at least one of the plurality of layers. The method includes identifying, misclassification of the input image in the neural network based on a result of the comparison.