Abstract: Described herein are means for integrating a defense mechanism into deep-learning-based systems to defend against adversarial attacks. For instance, an exemplary system is specially configured for adding a convolutinal defense layer to a neural network containing orthogonal kernels. Such a system generates the convolutional defense layer based on generating a set of learned kernals to increase diversity of network architecture, in which generating the set of learned kernals includes feeding an output of the convolutional defense layer into the neural network, further in which generating the convolutional defense layer includes selecting one or more orthogonal kernals, duplicating as needed and arranging them in a particular order.