Abstract: Techniques are described herein for using artificial intelligence (AI) and machine learning (ML) to automate, accelerate, and enhance various aspects of comparative user experience testing. Embodiments interface with generative language models to compare user experiences and summarize the results of the comparison. In some embodiments, automated systems and programmatic processes access a series of analysis contexts, where a context includes a collection of message content fragments. The systems and processes may use the message content fragments for a given context to construct a dialogue with a generative language model to compare separate user experiences based on the results of a set of user experience tests. The output of the generative language model at one stage of the analysis may be combined with content fragments for another context to craft a dialogue at another stage of the analysis and/or to perform additional analyses of the user experiences.

Abstract: Techniques are described herein for selecting, curating, normalizing, enriching, and synthesizing the results of user experience (UX) tests. In some embodiments, a system receives input defining or modifying a theme schema for classifying results of user experience tests. Responsive to receiving the input, the system trains a themer model based at least in part on example classifications in a training dataset, where the classifications map results to themes within the theme schema. When a new set of results for a user experience test is received, the trained machine learning model may generate a set of predicted themes to classify the test results. The output of the model may be used to render user interfaces and/or trigger other actions directed to optimizing a product's design.

Abstract: Techniques are described for producing machine-generating findings associated with user experiences with products and/or services. In some embodiments, a finding generator receives a set of user experience test results, generates a set of permutations of user attribute values, and, for each permutation, determines distributions of quantitative values that measure one or more facets of the user experience with a product or service for users that have the user attribute values and users that do not. Based on a comparison of the distributions, the finding generator identifies a subset of permutations to retain, generates segments of user experience test results based on the permutations, and generates findings summaries based on the results included in each segment. The findings may be presented to an analyst and or consumed by downstream application to perform actions directed at improving the design of the product or service.

Abstract: Techniques are described herein for using artificial intelligence (AI) and machine learning (ML) to automate, accelerate, and enhance various aspects of comparative user experience testing. Embodiments interface with generative language models to compare user experiences and summarize the results of the comparison. In some embodiments, automated systems and programmatic processes access a series of analysis contexts, where a context includes a collection of message content fragments. The systems and processes may use the message content fragments for a given context to construct a dialogue with a generative language model to compare separate user experiences based on the results of a set of user experience tests. The output of the generative language model at one stage of the analysis may be combined with content fragments for another context to craft a dialogue at another stage of the analysis and/or to perform additional analyses of the user experiences.

Abstract: Techniques are described herein for producing machine-generated findings given a set of user experience test results. In some embodiments, the system generates the findings using an artificial intelligence and machine learning engine. The findings may highlight areas that are predicted to provide the most insight into optimizing a product's design. A finding may be generated based on all or a subset of the test result elements, including qualitative and/or quantitative data contained therein. A finding may summarize a subset of the UX test results that are interrelated. A finding may link a summary to one or more references extracted from the set of test results to show support for the machine-generated insights in the underlying raw test data. Machine-generated findings reports may provide near instantaneous guidance for optimizing product designs while removing extraneous information from a vast quantity of raw test result data.

Abstract: Techniques are described herein for using artificial intelligence to select, curate, normalize, enrich, and synthesize the results of user experience (UX) tests. In some embodiments, a system identifies a set of unstructured textual elements associated with one or more UX tests. The system may configure agents using generative language model services, including a reviewing agent that reviews and edit outputs of a machine learning classification model applied to the unstructured textual elements and/or a curating agent that selects unstructured textual elements to represent themes within the user experience test classified using the machine learning classification model. The outputs may be used to enhance the scalability, function, and efficiency of applications directed at improving product designs.

Abstract: Techniques are described herein for selecting, curating, normalizing, enriching, and synthesizing the results of user experience (UX) tests. In some embodiments, a system receives input defining or modifying a theme schema for classifying results of user experience tests. Responsive to receiving the input, the system trains a themer model based at least in part on example classifications in a training dataset, where the classifications map results to themes within the theme schema. When a new set of results for a user experience test is received, the trained machine learning model may generate a set of predicted themes to classify the test results. The output of the model may be used to render user interfaces and/or trigger other actions directed to optimizing a product's design.

Abstract: Techniques are described herein for selecting, curating, normalizing, enriching, and synthesizing the results of user experience (UX) tests. In some embodiments, a system identifies a set of expectation elements associated with one or more UX tests. An expectation element may specify, using unstructured data that does not conform to a schema, an expectation for a user experience and a respective outcome for the user experience. A themer model may generate predictions that map the respective expectation elements to a theme from a theme schema, which may include a plurality of themes. A selector model may generate selection scores for the expectation elements. The predicted themes and selection scores may be used to render user interfaces and/or trigger other actions directed to optimizing a product's design.

Abstract: Techniques are described herein for selecting, curating, normalizing, enriching, and synthesizing the results of user experience (UX) tests. In some embodiments, a system receives input defining or modifying a theme schema for classifying results of user experience tests. Responsive to receiving the input, the system trains a themer model based at least in part on example classifications in a training dataset, where the classifications map results to themes within the theme schema. When a new set of results for a user experience test is received, the trained machine learning model may generate a set of predicted themes to classify the test results. The output of the model may be used to render user interfaces and/or trigger other actions directed to optimizing a product's design.

Abstract: Techniques are described for producing machine learning models to generate findings associated with user experiences with products and/or services. In some embodiments, a training process receives a set of findings from one or more user experience tests, where a finding includes a summary and a set of one or more references supporting the summary. The training process further identifies a supplemental set of one or more references that were not included in the initial finding to support the summary. The training process trains a machine learning model, such as a neural or generative language model, based on the first set of one or more references and the second set of one or more references to generate summaries from a subset of sampled references based at least in part on the first set of one or more references and the second set of one or more references.

Abstract: Techniques are described herein for selecting, curating, normalizing, enriching, and synthesizing the results of user experience (UX) tests. In some embodiments, a system identifies a set of expectation elements associated with one or more UX tests. An expectation element may specify, using unstructured data that does not conform to a schema, an expectation for a user experience and a respective outcome for the user experience. A themer model may generate predictions that map the respective expectation elements to a theme from a theme schema, which may include a plurality of themes. A selector model may generate selection scores for the expectation elements. The predicted themes and selection scores may be used to render user interfaces and/or trigger other actions directed to optimizing a product's design.