Abstract: Methods, systems, and techniques for kernel continuing learning. A dataset is obtained that corresponds to a classification task. Feature extraction is performed on the dataset using an artificial neural network. A kernel is constructed using features extracted during that feature extraction for use in performing the classification task. More particularly, during training, a coreset dataset corresponding to the classification task is saved; and during subsequent inference, the coreset dataset is retrieved and used to construct a task-specific kernel for classification.