Abstract: Described systems and methods allow the reduction of noise found in a corpus used for training automatic classifiers for anti-malware applications. Some embodiments target pairs of records, which have opposing labels, e.g. one record labeled as clean/benign, while the other labeled as malware. When two such records are found to be similar, they are identified as noise and are either discarded from the corpus, or relabeled. Two records may be deemed similar when, in a simple case, they share a majority of features, or, in a more sophisticated case, they are sufficiently close in a feature space according to some distance measure.