Abstract: A system and method for generating gold questions for labeling tasks are disclosed. The method includes sampling a positive class from a predefined set of classes to be used in labeling documents, based on a computed measure of class popularity. A set of negative classes is identified from the set of classes based on a distance measure between the positive class and other classes in the set of classes. A gold question is generated which includes a document representative of the positive class and a set of candidate answers. The candidate answers include a label for the positive class and a label for each of the negative classes in the identified set of negative classes. A task may be generated which includes the gold question and a plurality of standard questions which each include a document to be labeled. A computer processor may implement all or part of the method.

Abstract: The disclosed embodiments illustrate methods and systems for monitoring a worker performing a crowdsourced task being presented on a computing device. The method comprises performing at least one of a facial detection processing or an eye tracking processing, on a video stream captured by a camera of the computing device. An inattention instance of the worker is determined based on at least one of the facial detection processing or the eye tracking processing. Further, the inattention instance is communicated to a crowdsourcing platform, wherein the crowdsourced task is received from the crowdsourcing platform.

Abstract: A system and method of rendering in real time a virtual object onto a viewfinder display, the method comprising determining one or more scene properties of a scene on a viewfinder display of a device, receiving a virtual object for insertion into the scene, determining a location for placing the virtual object within the scene, determining a first appearance of the virtual object based on the one or more scene properties, and inserting the virtual object with the first appearance into the scene depicted on the viewfinder display of the device based on the location.