Abstract: Systems and methods for crowdsourced machine learning in accordance with embodiments of the invention are illustrated. In many embodiments, particular crowdworkers from a plurality of crowdworkers who are able to perform with high accuracy and reliability (referred to herein as “super recognizers”) are identified and used to generate training data for machine learning models. In various embodiments, super recognizers are identified by providing a request to answer questions regarding a particular type of input to the plurality of crowdworkers and providing received answers to a machine learning model trained using expert-annotated inputs similar to the inputs provided to the crowdworkers.