Abstract: A system, method and computer program product for training a deep neural network model using a pre-trained teacher model. Student training data samples are input to the deep neural network model and teacher training data samples are input to the pre-trained teacher model. The trained deep neural network model is generated using the training data samples to optimize an error function that is evaluated using a plurality of student label prediction outputs and a plurality of Markov transformed teacher label prediction outputs. Each Markov transformed teacher label prediction output is generated based on a teacher label prediction output by the pre-trained teacher model in response to receiving one of the teacher training data samples as an input. Each Markov transformed teacher label prediction output is generated through a Markov transform involving matrix multiplication using a Markov matrix.

Abstract: A system, method and computer program product for training a deep neural network. The deep neural network can be trained using a learning process that is defined to optimize both an error function of the deep neural network as well as a network mapping function of the deep neural network. The network mapping function can represent a predicted label distribution geometry property of the deep neural network. This learning process can improve the accuracy of the trained deep neural network model as well as its robustness again adversarial attacks. Optimizing the network mapping function can also provide increased insight into the operation of the trained deep neural network model, which may promote increased interpretability of the trained model and thus encourage uptake of the trained model.