Abstract: Techniques are described herein for selecting, curating, normalizing, enriching, and synthesizing the results of user experience tests. In some embodiments, a system identifies a qualitative element within a result set for a user experience test. The system then selects a machine learning model to apply based on one or more attributes associated with the user experience test and generates a predicted visibility, quality, and/or relevance for the qualitative element. Based on the prediction, the system generates a user interface that curates a set of results of the user experience test.

Abstract: Techniques are disclosed for summarizing, diagnosing, and correcting the cause of anomalous behavior in computing systems. In some embodiments, a system identifies a plurality of time series that track different metrics over time for a set of one or more computing resources. The system detects a first set of anomalies in a first time series that tracks a first metric and assigns a different respective range of time to each anomaly. The system determines whether the respective range of time assigned to an anomaly overlaps with timestamps or ranges of time associated with anomalies from one or more other time series. The system generates at least one cluster that groups metrics based on how many anomalies have respective ranges of time and/or timestamps that overlap. The system may preform, based on the cluster, one or more automated actions for diagnosing or correcting a cause of anomalous behavior.

Abstract: Techniques are described for automatically detecting and accommodating state changes in a computer-generated forecast. In one or more embodiments, a representation of a time-series signal is generated within volatile and/or non-volatile storage of a computing device. The representation may be generated in such a way as to approximate the behavior of the time-series signal across one or more seasonal periods. Once generated, a set of one or more state changes within the representation of the time-series signal is identified. Based at least in part on at least one state change in the set of one or more state changes, a subset of values from the sequence of values is selected to train a model. An analytical output is then generated, within volatile and/or non-volatile storage of the computing device, using the trained model.

Abstract: Techniques for predictive system remediation are disclosed. Based on attributes associated with applications of one or more system-selected remedial actions to one or more problematic system behaviors in a system (e.g., a database system), the system determines a predicted effectiveness of one or more future applications of a remedial action to a particular problematic system behavior, as of one or more future times. The system determines that the predicted effectiveness of the one or more future applications of the remedial action is positive but does not satisfy a performance criterion. Responsive to determining that the predicted effectiveness is positive but does not satisfy the performance criterion, the system generates a notification corresponding to the predicted effectiveness not satisfying the performance criterion.

Abstract: Techniques are described for classifying seasonal patterns in a time series. In an embodiment, a set of time series data is decomposed to generate a noise signal and a dense signal, where the noise signal includes a plurality of sparse features from the set of time series data and the dense signal includes a plurality of dense features from the set of time series data. A set of one or more sparse features from the noise signal is selected for retention. After selecting the sparse features, a modified set of time series data is generated by combining the set of one or more sparse features with a set of one or more dense features from the plurality of dense features. At least one seasonal pattern is identified from the modified set of time series data. A summary for the seasonal pattern may then be generated and stored.

Abstract: Techniques are described herein for selecting, curating, normalizing, enriching, and synthesizing the results of user experience tests. In some embodiments, a system identifies a qualitative element within a result set for a user experience test. The system then selects a machine learning model to apply based on one or more attributes associated with the user experience test and generates a predicted visibility, quality, and/or relevance for the qualitative element. Based on the prediction, the system generates a user interface that curates a set of results of the user experience test.

Abstract: Techniques are described for producing machine learning models to generate findings associated with user experiences with products and/or services. In some embodiments, a training process receives a set of findings from one or more user experience tests, where a finding includes a summary and a set of one or more references supporting the summary. The training process further identifies a supplemental set of one or more references that were not included in the initial finding to support the summary. The training process trains a machine learning model, such as a neural or generative language model, based on the first set of one or more references and the second set of one or more references to generate summaries from a subset of sampled references based at least in part on the first set of one or more references and the second set of one or more references.

Abstract: Techniques are described herein for selecting, curating, normalizing, enriching, and synthesizing the results of user experience (UX) tests. In some embodiments, a system identifies a set of expectation elements associated with one or more UX tests. An expectation element may specify, using unstructured data that does not conform to a schema, an expectation for a user experience and a respective outcome for the user experience. A themer model may generate predictions that map the respective expectation elements to a theme from a theme schema, which may include a plurality of themes. A selector model may generate selection scores for the expectation elements. The predicted themes and selection scores may be used to render user interfaces and/or trigger other actions directed to optimizing a product's design.

Abstract: Generating persistent multifaceted statistical distributions of event data associated with computing nodes is disclosed. From a data stream, events are identified that occur during a first time interval. Characteristics associated with the events are determined. Based on a primary characteristic, it is determined that an event corresponds to an event cluster. The event count for that cluster is incremented. It is determined that the characteristics correspond to an event descriptor of events in the cluster. Responsive to requests to view the event cluster, information about descriptors from the cluster are displayed indicating events having a particular event descriptor, or a summary of characteristics that distinguish the descriptor from other event descriptors.

Abstract: Techniques are described for generating seasonal forecasts. According to an embodiment, a set of time-series data is associated with one or more classes, which may include a first class that represent a dense pattern that repeats over multiple instances of a season in the set of time-series data and a second class that represent another pattern that repeats over multiple instances of the season in the set of time-series data. A particular class of data is associated with at least two sub-classes of data, where a first sub-class represents high data points from the first class, and a second sub-class represents another set of data points from the first class. A trend rate is determined for a particular sub-class. Based at least in part on the trend rate, a forecast is generated.

Abstract: Techniques are disclosed for summarizing, diagnosing, and correcting the cause of anomalous behavior in computing systems. In some embodiments, a system identifies a plurality of time series that track different metrics over time for a set of one or more computing resources. The system detects a first set of anomalies in a first time series that tracks a first metric and assigns a different respective range of time to each anomaly. The system determines whether the respective range of time assigned to an anomaly overlaps with timestamps or ranges of time associated with anomalies from one or more other time series. The system generates at least one cluster that groups metrics based on how many anomalies have respective ranges of time and/or timestamps that overlap. The system may preform, based on the cluster, one or more automated actions for diagnosing or correcting a cause of anomalous behavior.

Abstract: Systems and methods for unsupervised training and evaluation of anomaly detection models are described. In some embodiments, an unsupervised process comprises generating an approximation of a data distribution for a training dataset including varying values for a metric of a computing resource. The process further determines, based on the size of the training dataset, a first quantile probability and a second quantile probability that represent an interval for covering a prescribed proportion of values for the metric within a prescribed confidence level. The process further trains a lower limit of the anomaly detection model using a first quantile that represents the first quantile probability in the approximation of the data distribution and an upper limit using a second quantile that represents the second quantile probability in the approximation. The trained upper and lower limits may be used to monitor input data for anomalous behavior and, if detected, trigger responsive action(s).

Abstract: Techniques are disclosed for summarizing, diagnosing, and correcting the cause of anomalous behavior in computing systems. In some embodiments, a system identifies a plurality of time series that track different metrics over time for a set of one or more computing resources. The system detects a first set of anomalies in a first time series that tracks a first metric and assigns a different respective range of time to each anomaly. The system determines whether the respective range of time assigned to an anomaly overlaps with timestamps or ranges of time associated with anomalies from one or more other time series. The system generates at least one cluster that groups metrics based on how many anomalies have respective ranges of time and/or timestamps that overlap. The system may preform, based on the cluster, one or more automated actions for diagnosing or correcting a cause of anomalous behavior.

Abstract: Techniques for predictive system remediation are disclosed. Based on attributes associated with applications of one or more system-selected remedial actions to one or more problematic system behaviors in a system (e.g., a database system), the system determines a predicted effectiveness of one or more future applications of a remedial action to a particular problematic system behavior, as of one or more future times. The system determines that the predicted effectiveness of the one or more future applications of the remedial action is positive but does not satisfy a performance criterion. Responsive to determining that the predicted effectiveness is positive but does not satisfy the performance criterion, the system generates a notification corresponding to the predicted effectiveness not satisfying the performance criterion.

Abstract: Techniques for predictive system remediation are disclosed. Based on attributes associated with applications of one or more system-selected remedial actions to one or more problematic system behaviors in a system (e.g., a database system), the system determines a predicted effectiveness of one or more future applications of a remedial action to a particular problematic system behavior, as of one or more future times. The system determines that the predicted effectiveness of the one or more future applications of the remedial action is positive but does not satisfy a performance criterion. Responsive to determining that the predicted effectiveness is positive but does not satisfy the performance criterion, the system generates a notification corresponding to the predicted effectiveness not satisfying the performance criterion.

Abstract: Techniques are described for characterizing and summarizing seasonal patterns detected within a time series. According to an embodiment, a set of time series data is analyzed to identify a plurality of instances of a season, where each instance corresponds to a respective sub-period within the season. A first set of instances from the plurality of instances are associated with a particular class of seasonal pattern. After classifying the first set of instances, a second set of instances may remain unclassified or otherwise may not be associated with the particular class of seasonal pattern. Based on the first and second set of instances, a summary may be generated that identifies one or more stretches of time that are associated with the particular class of seasonal pattern. The one or more stretches of time may span at least one sub-period corresponding to at least one instance in the second set of instances.

Abstract: Systems and methods for trending patterns within a set of time-series data are described. In one or more embodiments, a set of one or more groups of data points that are associated with a particular seasonal pattern are generated within volatile and/or non-volatile storage. A set of pairwise slopes is determined for data point pairs within the set of one or more groups of data points. Based, at least in part on the plurality of pairwise slopes, a representative trend rate for the particular seasonal pattern is determined. A set of forecasted values is then generated within volatile or non-volatile storage based, at least in part, on the representative trend rate for the particular seasonal pattern.

Abstract: Generating persistent multifaceted statistical distributions of event data associated with computing nodes is disclosed. From a data stream, events are identified that occur during a first time interval. Characteristics associated with the events are determined. Based on a primary characteristic, it is determined that an event corresponds to an event cluster. The event count for that cluster is incremented. It is determined that the characteristics correspond to an event descriptor of events in the cluster. Responsive to requests to view the event cluster, information about descriptors from the cluster are displayed indicating events having a particular event descriptor, or a summary of characteristics that distinguish the descriptor from other event descriptors.

Abstract: Systems and methods for performing unsupervised baselining and anomaly detection using time-series data are described. In one or more embodiments, a baselining and anomaly detection system receives a set of time-series data. Based on the set of time-series, the system generates a first interval that represents a first distribution of sample values associated with the first seasonal pattern and a second interval that represents a second distribution of sample values associated with the second seasonal pattern. The system then monitors a time-series signals using the first interval during a first time period and the second interval during a second time period. In response to detecting an anomaly in the first seasonal pattern or the second seasonal pattern, the system performs a responsive action, such as generating an alert.

Abstract: Techniques for training and evaluating seasonal forecasting models are disclosed. In some embodiments, a network service generates, in memory, a set of data structures that separate sample values by season type and season space. The set of data structures may include a first set of clusters corresponding to different season types in the first season space and a second set of clusters corresponding to different season types in the second season space. The network service merges two or more clusters the first set and/or second set of clusters. Clusters from the first set are not merged with clusters from the second set. After merging the clusters, the network service determines a trend pattern for each of the remaining clusters in the first and second set of clusters. The network service then generates a forecast for a metric of a computing resource based on the trend patterns for each remaining cluster.