Abstract: Techniques are described herein for testing and characterizing user experiences with digital assets. Embodiments include a digital experience testing system for running usability tests with respect to user interface designs. The runtime testing environment may use a set of one or more compiled prototypes to test various facets of a user experience including what respondents do when interacting with a digital asset and how the experience engages the respondents' emotions. The digital experience testing system may analyze, enrich, synthesize, and/or otherwise process the results of one or more usability tests, based on the tracked user interactions, to provide guidance on user interface design optimizations.

Abstract: Operations of a prompt management system are disclosed. The operations may include: receiving a prompt for performing a set of tasks, assigning an agent group that includes a plurality of agents to perform a set of roles associated with a dataset in support of the set of tasks, causing the plurality of agents to perform the set of roles using a first machine-learning model, receiving a set of role results from the plurality of agents responsive to performing the set of roles, performing the set of tasks using at least a second machine-learning model, and providing a task result for display on a user interface device. The set of tasks may include executing an operation on the set of role results using the second machine-learning model, and generating a task result that includes a product of the operation executed on the set of role results.

Abstract: Operations of a prompt management system are disclosed. The operations may include: receiving a prompt for performing a set of tasks, assigning an agent group that includes a plurality of agents to perform a set of roles associated with a dataset in support of the set of tasks, causing the plurality of agents to perform the set of roles using a first machine-learning model, receiving a set of role results from the plurality of agents responsive to performing the set of roles, performing the set of tasks using at least a second machine-learning model, and providing a task result for display on a user interface device. The set of tasks may include executing an operation on the set of role results using the second machine-learning model, and generating a task result that includes a product of the operation executed on the set of role results.

Abstract: Techniques and embodiments described herein include a scalable system for integrating panel-based research with user experience (UX) testing tools, such as online survey applications. The techniques provide for the parallelization of a single request across multiple panel providers. The parallelization across multiple panel providers may occur transparently to users, including the designers of UX testing tools and methodologies, without requiring any complex modifications of the underlying source code. The parallelization may further significantly reduce request processing speeds within the system. Additionally or alternatively, the techniques provide for runtime adjustments of configurations to adjust rates at which respondents with varying attributes are fielded. As a result, qualified UX test respondents may be fielded much more quickly, increasing system scalability by allowing the system to process more requests within a given timeframe.

Abstract: Techniques are described herein for using artificial intelligence to select, curate, normalize, enrich, and synthesize the results of user experience (UX) tests. In some embodiments, a system identifies a set of unstructured textual elements associated with one or more UX tests. The system may configure agents using generative language model services, including a reviewing agent that reviews and edit outputs of a machine learning classification model applied to the unstructured textual elements and/or a curating agent that selects unstructured textual elements to represent themes within the user experience test classified using the machine learning classification model. The outputs may be used to enhance the scalability, function, and efficiency of applications directed at improving product designs.

Abstract: Techniques and embodiments are described herein for detecting and mitigating fraudulent activity within user experience (UX) test applications. In some embodiments, a system applies a set of rules and/or machine learning (ML) models to each respondent of an online survey or UX test. Different ML models may be trained to learn domain-specific patterns indicative of fraudulent activity. The system may then select the ML models based on attributes of the UX test and/or respondent. The selected rules and/or ML models may generate a probabilistic score representing a likelihood that the respondent is currently engaging in or will engage in fraudulent activity with respect to a UX test. If the score exceeds a threshold, then the system may take action to mitigate the fraudulent activity, such as triggering the removal of the user from an accepted respondent pool, halting further engagement between the respondent and the UX test, and generating alerts.

Abstract: Techniques are described herein for selecting, curating, normalizing, enriching, and synthesizing the results of user experience tests. In some embodiments, a system identifies a qualitative element within a result set for a user experience test. The system then selects a machine learning model to apply based on one or more attributes associated with the user experience test and generates a predicted visibility, quality, and/or relevance for the qualitative element. Based on the prediction, the system generates a user interface that curates a set of results of the user experience test.

Abstract: Techniques are described herein for selecting, curating, normalizing, enriching, and synthesizing the results of user experience tests. In some embodiments, a system identifies a qualitative element within a result set for a user experience test. The system then selects a machine learning model to apply based on one or more attributes associated with the user experience test and generates a predicted visibility, quality, and/or relevance for the qualitative element. Based on the prediction, the system generates a user interface that curates a set of results of the user experience test.

Abstract: Techniques are described herein for providing adaptable testing and benchmarking of user experiences with respect to one or more products. In some embodiments, the techniques include systems and methods for predicting performance of facets of a user experience under new testing and benchmarking methodologies. The systems and methods may generate a prediction for the results of a UX test even though the methodology and mechanics to quantify the user experience may vary significantly from previous methodologies. The techniques allow for methodologies to evolve over time without losing historical context or the ability to meaningfully compare historical test results with tests run using updated testing and benchmark models. Further, the techniques allow for benchmarks to be computed in real-time or near real-time as methodologies change without requiring tests to be run according to the new methodologies.