Abstract: Techniques are described herein for selecting, curating, normalizing, enriching, and synthesizing the results of user experience (UX) tests. In some embodiments, a system receives input defining or modifying a theme schema for classifying results of user experience tests. Responsive to receiving the input, the system trains a themer model based at least in part on example classifications in a training dataset, where the classifications map results to themes within the theme schema. When a new set of results for a user experience test is received, the trained machine learning model may generate a set of predicted themes to classify the test results. The output of the model may be used to render user interfaces and/or trigger other actions directed to optimizing a product's design.

Abstract: Techniques are described herein for selecting, curating, normalizing, enriching, and synthesizing the results of user experience (UX) tests. In some embodiments, a system receives input defining or modifying a theme schema for classifying results of user experience tests. Responsive to receiving the input, the system trains a themer model based at least in part on example classifications in a training dataset, where the classifications map results to themes within the theme schema. When a new set of results for a user experience test is received, the trained machine learning model may generate a set of predicted themes to classify the test results. The output of the model may be used to render user interfaces and/or trigger other actions directed to optimizing a product's design.

Abstract: Techniques are described herein for selecting, curating, normalizing, enriching, and synthesizing the results of user experience (UX) tests. In some embodiments, a system identifies a set of expectation elements associated with one or more UX tests. An expectation element may specify, using unstructured data that does not conform to a schema, an expectation for a user experience and a respective outcome for the user experience. A themer model may generate predictions that map the respective expectation elements to a theme from a theme schema, which may include a plurality of themes. A selector model may generate selection scores for the expectation elements. The predicted themes and selection scores may be used to render user interfaces and/or trigger other actions directed to optimizing a product's design.

Abstract: Techniques and embodiments are described herein for detecting and mitigating fraudulent activity within user experience (UX) test applications. In some embodiments, a system applies a set of rules and/or machine learning (ML) models to each respondent of an online survey or UX test. Different ML models may be trained to learn domain-specific patterns indicative of fraudulent activity. The system may then select the ML models based on attributes of the UX test and/or respondent. The selected rules and/or ML models may generate a probabilistic score representing a likelihood that the respondent is currently engaging in or will engage in fraudulent activity with respect to a UX test. If the score exceeds a threshold, then the system may take action to mitigate the fraudulent activity, such as triggering the removal of the user from an accepted respondent pool, halting further engagement between the respondent and the UX test, and generating alerts.

Abstract: Techniques are described herein for selecting, curating, normalizing, enriching, and synthesizing the results of user experience tests. In some embodiments, a system identifies a qualitative element within a result set for a user experience test. The system then selects a machine learning model to apply based on one or more attributes associated with the user experience test and generates a predicted visibility, quality, and/or relevance for the qualitative element. Based on the prediction, the system generates a user interface that curates a set of results of the user experience test.

Abstract: Techniques are described herein for selecting, curating, normalizing, enriching, and synthesizing the results of user experience tests. In some embodiments, a system identifies a qualitative element within a result set for a user experience test. The system then selects a machine learning model to apply based on one or more attributes associated with the user experience test and generates a predicted visibility, quality, and/or relevance for the qualitative element. Based on the prediction, the system generates a user interface that curates a set of results of the user experience test.

Abstract: Techniques are described for producing machine learning models to generate findings associated with user experiences with products and/or services. In some embodiments, a training process receives a set of findings from one or more user experience tests, where a finding includes a summary and a set of one or more references supporting the summary. The training process further identifies a supplemental set of one or more references that were not included in the initial finding to support the summary. The training process trains a machine learning model, such as a neural or generative language model, based on the first set of one or more references and the second set of one or more references to generate summaries from a subset of sampled references based at least in part on the first set of one or more references and the second set of one or more references.

Abstract: Techniques are described herein for selecting, curating, normalizing, enriching, and synthesizing the results of user experience (UX) tests. In some embodiments, a system identifies a set of expectation elements associated with one or more UX tests. An expectation element may specify, using unstructured data that does not conform to a schema, an expectation for a user experience and a respective outcome for the user experience. A themer model may generate predictions that map the respective expectation elements to a theme from a theme schema, which may include a plurality of themes. A selector model may generate selection scores for the expectation elements. The predicted themes and selection scores may be used to render user interfaces and/or trigger other actions directed to optimizing a product's design.

Abstract: Techniques are described herein for providing adaptable testing and benchmarking of user experiences with respect to one or more products. In some embodiments, the techniques include systems and methods for predicting performance of facets of a user experience under new testing and benchmarking methodologies. The systems and methods may generate a prediction for the results of a UX test even though the methodology and mechanics to quantify the user experience may vary significantly from previous methodologies. The techniques allow for methodologies to evolve over time without losing historical context or the ability to meaningfully compare historical test results with tests run using updated testing and benchmark models. Further, the techniques allow for benchmarks to be computed in real-time or near real-time as methodologies change without requiring tests to be run according to the new methodologies.