Abstract: Generally discussed herein are devices, systems, and methods for generating a phrase that is confusing to a language classifier. A method can include determining, by the LC, a first classification score (CS) of a prompt indicating whether the prompt is a first class or a second class, predicting, based on the prompt and by a pre-trained language model (PLM), likely next words and a corresponding probability for each of the likely next words, determining, by the LC, a second CS for each of the likely next words, determining, by an adversarial classifier, respective scores for each of the likely next words, the respective scores determined based on the first CS of the prompt, the second CS of the likely next words, and the probabilities of the likely next words, and selecting, by an adversarial classifier, a next word of the likely next words based on the respective scores.

Abstract: The present disclosure relates to systems, methods, and computer readable media that evaluate performance of a machine learning system in connection with a test dataset. For example, systems disclosed herein may receive a test dataset and identify label information for the test dataset including feature information and ground truth data. The systems disclosed herein can compare the ground truth data and outputs generated by a machine learning system to evaluate performance of the machine learning system with respect to the test dataset. The systems disclosed herein may further generate feature clusters based on failed outputs and corresponding features and generate a number of performance views that illustrate performance of the machine learning system with respect to clustered groupings of the test dataset.

Abstract: The present disclosure relates to systems, methods, and computer readable media that evaluate performance of a machine learning system in connection with a test dataset. For example, systems disclosed herein may receive a test dataset and identify label information for the test dataset including feature information and ground truth data. The systems disclosed herein can compare the ground truth data and outputs generated by a machine learning system to evaluate performance of the machine learning system with respect to the test dataset. The systems disclosed herein may further generate feature clusters based on failed outputs and corresponding features and generate a number of performance views that illustrate performance of the machine learning system with respect to clustered groupings of the test dataset.

Abstract: Edits on a content item, such as a document, are divided into microtasks. The microtasks associated with a document can be automatically identified based on a workflow or can be identified by a user associated with the content item or an administrator. At a later time, the user can complete the microtasks for a content item using an application associated with their smart phone or tablet. The application may present the microtasks in a game-like environment where the user can compete with other users based on metrics such as number of microtasks completed in a day or fastest completion time. In addition, the user can earn rewards such as badges, coupons, or credits by completing microtasks. In this way, users can use time that would have been wasted playing games to complete their content items, while still experiencing some of the fun and competition associated with the games.

Abstract: The present disclosure relates to systems, methods and computer readable media for evaluating performance of a machine learning system and providing one or more performance views representative of the determined performance. For example, systems disclosed herein may receive or identify performance information including outputs, accuracy data, and feature data associated with a plurality of test instances. In addition, systems disclosed herein may provide one or more performance views via a graphical user interface including graphical elements (e.g., interactive elements) and indications of accuracy data and other performance data with respect to feature clusters associated with select groupings of test instances from the plurality of test instances. The performance views may include interactive features to enable a user to view and intuitively understand performance of the machine learning system with respect to clustered groupings of test instances that share common characteristics.

Abstract: Edits on a content item, such as a document, are divided into microtasks. The microtasks associated with a document can be automatically identified based on a workflow or can be identified by a user associated with the content item or an administrator. At a later time, the user can complete the microtasks for a content item using an application associated with their smart phone or tablet. The application may present the microtasks in a game-like environment where the user can compete with other users based on metrics such as number of microtasks completed in a day or fastest completion time. In addition, the user can earn rewards such as badges, coupons, or credits by completing microtasks. In this way, users can use time that would have been wasted playing games to complete their content items, while still experiencing some of the fun and competition associated with the games.

Abstract: A stochastic privacy service provider may provide users with a guaranteed upper bound on a probability that personal data will be accessed while enabling the services to collect data that can be used to enhance its services. Users may receive incentives to become participants in a stochastic privacy program. The stochastic privacy provider may employ one or more probabilistic and decision-theoretic methods to determine which participants' personal data should be sought while guaranteeing that the probability of personal data being accessed is smaller than the mutually agreed upon probability of access. The probability of access may be on a per time basis. The stochastic privacy provider may access coalescenses of the personal data of sets of multiple people, where a maximum probability is given for accessing statistical summaries of personal data computed from groups of people that are of at least some guaranteed size.

Abstract: Various technologies pertaining to dynamically identifying travel segments to be taken by a traveler traveling in a region are described herein, where observations about travel segments in the region are sparse and subject to alteration. A computer-implemented graph can be loaded into a memory, where the computer-implemented graph is representative of the region. The computer-implemented graph includes nodes that represent locations in the region and edges that represent travel segments of the region, where the edges have costs assigned thereto, and further where there is a defined statistical relationship between the costs. When an observation about a travel path is received, using the computer-implemented graph, inferences can be made about costs of traversing other travel paths in the region.

Abstract: Edits on a content item, such as a document, are divided into microtasks. The microtasks associated with a document can be automatically identified based on a workflow or can be identified by a user associated with the content item or an administrator. At a later time, the user can complete the microtasks for a content item using an application associated with their smart phone or tablet. The application may present the microtasks in a game-like environment where the user can compete with other users based on metrics such as number of microtasks completed in a day or fastest completion time. In addition, the user can earn rewards such as badges, coupons, or credits by completing microtasks. In this way, users can use time that would have been wasted playing games to complete their content items, while still experiencing some of the fun and competition associated with the games.

Abstract: Edits on a content item, such as a document, are divided into microtasks. The microtasks associated with a document can be automatically identified based on a workflow or can be identified by a user such as the creator of the content item or an administrator. The microtasks can be assigned to one or more workers including the creator of the content item. When a determination is made that an assigned worker is available to complete a microtask (e.g., when the worker is waiting in line, has just closed an application or file, or has just completed a phone call, etc.), the assigned microtask is presented to the worker for completion.

Abstract: Various technologies pertaining to dynamically identifying travel segments to be taken by a traveler traveling in a region are described herein, where observations about travel segments in the region are sparse and subject to alteration. A computer-implemented graph can be loaded into a memory, where the computer-implemented graph is representative of the region. The computer-implemented graph includes nodes that represent locations in the region and edges that represent travel segments of the region, where the edges have costs assigned thereto, and further where there is a defined statistical relationship between the costs. When an observation about a travel path is received, using the computer-implemented graph, inferences can be made about costs of traversing other travel paths in the region.

Abstract: A crowdsourcing environment is described herein which uses a single-stage or multi-stage approach to evaluate the quality of work performed by a worker, with respect to an identified task. In the multi-stage case, an evaluation system, in the first stage, determines whether the worker corresponds to a spam agent. In a second stage, for a non-spam worker, the evaluation system determines the propensity of the worker to perform desirable (e.g., accurate) work in the future. The evaluation system operates based on a set of features, including worker-focused features (which describe work performed by the particular worker), task-focused features (which describe tasks performed in the crowdsourcing environment), and system-focused features (which describe aspects of the configuration of the crowdsourcing environment). According to one illustrative aspect, the evaluation system performs its analysis using at least one model, produced using any type of supervised machine learning technique.