Abstract: The invention provides a model training method and system that uses pretrained features of a teacher neural network trained on a billion-size dataset to train a student neural network. The model training method leverages the teacher neural network to design a more stable loss function that works well with more sophisticated learning rate schedules to reduce training time and make the augmentation designing process more natural.