Abstract: Methods, systems, and apparatus, including computer programs to detect anomalous patterns in training artificial neural networks in a computing system, begins by training a neural network in a supervised manner with labeled datasets divided into training, validation and test subsets. The neural network model includes a plurality of layers each having a plurality of parameters. The system saves checkpoints of the model during training that represents different versions of a partially trained machine learning model during different stages of training. The method searches for anomalous patterns between checkpoint versions, calculates a subset of parameters in each layer and returns the results. The search results can be used to modify the neural network model improving accuracy and loss on training, validation and tests dataset.