Abstract: In an embodiment, a process for automatic model monitoring for data streams includes receiving an input dataset, using a machine learning model to determine a model score for each data record of at least a portion of the input dataset, and determining monitoring values. Each monitoring value is associated with a measure of similarity between model scores for those data records of the input dataset within a corresponding moving reference window and model scores for those data records of the input dataset within a corresponding moving target window. The process includes outputting the determined monitoring values.

Abstract: In various embodiments, a process for providing a distributed streaming system supporting real-time sliding windows includes receiving a stream of events at a plurality of distributed nodes and routing the events into topic groupings. The process includes using one or more events in at least one of the topic groupings to determine one or more metrics of events with at least one window and an event reservoir including by: tracking, in a volatile memory of the event reservoir, beginning and ending events within the at least one window; and tracking, in a persistent storage of the event reservoir, events associated with tasks assigned to a respective node. The process includes updating the one or more metrics based on one or more previous values of the one or more metrics as a new event is added or an existing event is expired from the at least one window.

Abstract: In an embodiment, a process for automatic model monitoring for data streams includes receiving an input dataset, using a machine learning model to determine a model score for each data record of at least a portion of the input dataset, and determining monitoring values. Each monitoring value is associated with a measure of similarity between model scores for those data records of the input dataset within a corresponding moving reference window and model scores for those data records of the input dataset within a corresponding moving target window. The process includes outputting the determined monitoring values.

Abstract: In an embodiment, a process for adaptive threshold estimation for streaming data includes determining initial positions for a set of percentile bins, receiving a new data item in a stream of data, and identifying one of the set of percentile bins corresponding to the new data item. The process includes incrementing a count of items in the identified percentile bin, adjusting one or more counts of data items in one or more of the percentile bins including by applying a suppression factor based on a relative ordering of items, and redistributing positions for the set of percentile bins to equalize respective count numbers of items for each percentile bin of the set of percentile bins. The process includes utilizing the redistributed positions of the set of percentile bins to determine a percentile distribution of the data stream, and calculating a threshold based at least in part on the percentiles distribution.

Abstract: In an embodiment, a process for automatic model monitoring for data streams includes receiving an input dataset, using a machine learning model to determine a model score for each data record of at least a portion of the input dataset, and determining monitoring values. Each monitoring value is associated with a measure of similarity between model scores for those data records of the input dataset within a corresponding moving reference window and model scores for those data records of the input dataset within a corresponding moving target window. The process includes outputting the determined monitoring values.

Abstract: One or more events of a data stream are received. For each feature of a set of features, the one or more events are used to update a corresponding distribution of data from the data stream. For each feature of the set of features, the corresponding updated distribution and a corresponding reference distribution are used to determine a corresponding divergence value. For each feature of the set of features, the corresponding determined divergence value and a corresponding distribution of divergences are used to determine a corresponding statistical value. Using the statistical values each corresponding to a different feature of the set of features, a statistical analysis is performed to determine a result associated with a likelihood of data drift detection.

Abstract: A set of data elements is received. For each feature of a set of features, a corresponding reference distribution for the set of data elements is determined. For each feature of the set of features, one or more corresponding subset distributions for one or more subsets sampled from the set of data elements are determined. For each feature of the set of features, the corresponding reference distribution is compared with each of the one or more corresponding subset distributions to determine a corresponding distribution of divergences. At least the determined distributions of divergences for the set of features are provided for use in automated data analysis.

Abstract: In various embodiments, a process for providing a distributed streaming system supporting real-time sliding windows includes receiving a stream of events at a plurality of distributed nodes and routing the events into topic groupings. The process includes using one or more events in at least one of the topic groupings to determine one or more metrics of events with at least one window and an event reservoir including by: tracking, in a volatile memory of the event reservoir, beginning and ending events within the at least one window; and tracking, in a persistent storage of the event reservoir, events associated with tasks assigned to a respective node. The process includes updating the one or more metrics based on one or more previous values of the one or more metrics as a new event is added or an existing event is expired from the at least one window.

Abstract: In various embodiments, a process for providing a distributed streaming system supporting real-time sliding windows includes receiving a stream of events at a plurality of distributed nodes and routing the events into topic groupings. The process includes using one or more events in at least one of the topic groupings to determine one or more metrics of events with at least one window and an event reservoir including by: tracking, in a volatile memory of the event reservoir, beginning and ending events within the at least one window; and tracking, in a persistent storage of the event reservoir, all events associated with tasks assigned to a respective node. The process includes updating the one or more metrics based on one or more previous values of the one or more metrics as a new event is added or an existing event is expired from the at least one window.

Abstract: In various embodiments, a process for providing a distributed streaming system supporting real-time sliding windows includes receiving a stream of events at a plurality of distributed nodes and routing the events into topic groupings. The process includes using one or more events in at least one of the topic groupings to determine one or more metrics of events with at least one window and an event reservoir including by: tracking, in a volatile memory of the event reservoir, beginning and ending events within the at least one window; and tracking, in a persistent storage of the event reservoir, all events associated with tasks assigned to a respective node. The process includes updating the one or more metrics based on one or more previous values of the one or more metrics as a new event is added or an existing event is expired from the at least one window.

Abstract: In an embodiment, a process for adaptive threshold estimation for streaming data includes determining initial positions for a set of percentile bins, receiving a new data item in a stream of data, and identifying one of the set of percentile bins corresponding to the new data item. The process includes incrementing a count of items in the identified percentile bin, adjusting one or more counts of data items in one or more of the percentile bins including by applying a suppression factor based on a relative ordering of items, and redistributing positions for the set of percentile bins to equalize respective count numbers of items for each percentile bin of the set of percentile bins. The process includes utilizing the redistributed positions of the set of percentile bins to determine a percentile distribution of the data stream, and calculating a threshold based at least in part on the percentiles distribution.

Abstract: In an embodiment, a process for explanation reporting based on differentiation between items in different data groups includes obtaining model scores from a first machine learning model and training a second machine learning model to learn how to differentiate between two groups based on at least one of: features and the model scores obtained from the first machine learning model. The process includes applying the second machine learning model to each data record in a first group of data records to determine a corresponding ranking score for each data record in the first group, and based on the corresponding ranking scores, determining a relative contribution of each of the data records in the first group to the differentiation between the first group of data records and a second group of data records.

Abstract: In an embodiment, a process for automatic model monitoring for data streams includes receiving an input dataset, using a machine learning model to determine a model score for each data record of at least a portion of the input dataset, and determining monitoring values. Each monitoring value is associated with a measure of similarity between model scores for those data records of the input dataset within a corresponding moving reference window and model scores for those data records of the input dataset within a corresponding moving target window. The process includes outputting the determined monitoring values.

Abstract: In an embodiment, a process for semantic-aware feature engineering includes receiving semantic labels for data fields of training data. Each of the semantic labels is associated with a semantic meaning associated with a corresponding data field. The process includes automatically generating at least one new feature using at least a portion of the semantic labels.