Abstract: The subject disclosure relates to techniques for maintaining an optimized mix of samples selected from a smaller data set and a larger data set for use in training a machine learning algorithm. A process of the disclosed technology can include creating a plurality of chunks of samples, wherein each chunk contains a predetermined proportion of samples from the smaller data set and the larger data set, while also including substantially all samples in the larger dataset are distributed across the plurality of chunks of samples, loading a first chunk of samples of the plurality of chunks of samples into a memory, randomizing the order of samples in the first chunk of samples, and providing the samples in the first chunk of samples in the randomized order into the machine learning algorithm.