Abstract: Apparatus and methods for training classifiers. The apparatus includes a degree of certainty classifier which classifies examples in categories and indicates a degree of certainty regarding the classification and an annotating classifier which receives classified examples with a low degree of certainty from the degree of certainty classifier and annotates them to indicate whether their classifications are correct. The annotated examples are then used to train another classifier. In one version of the invention, the other classifier is a new version of the degree of certainty classifier, and training continues until the degree of certainty classifier has satisfactory performance. The degree of certainty classifier of the embodiment is a probabilistic binary classifier which is trained using relevance feedback. The annotating classifier may include an interactive interface which permits a human user of the system to examine an example and indicate whether it was properly classified.