Abstract: A predictive modeling method may include obtaining a fitted, first-order predictive model configured to predict values of output variables based on values of first input variables; and performing a second-order modeling procedure on the fitted, first-order model, which may include: generating input data including observations including observed values of second input variables and predicted values of the output variables; generating training data and testing data from the input data; generating a fitted second-order model of the fitted first-order model by fitting a second-order model to the training data; and testing the fitted, second-order model of the first-order model on the testing data. Each observation of the input data may be generated by (1) obtaining observed values of the second input variables, and (2) applying the first-order predictive model to corresponding observed values of the first input variables to generate the predicted values of the output variables.

Abstract: Automated data analytics techniques for non-tabular data sets may include methods and systems for (1) automatically developing models that perform tasks in the domains of computer vision, audio processing, speech processing, text processing, or natural language processing; (2) automatically developing models that analyze heterogeneous data sets containing image data and non-image data, and/or heterogeneous data sets containing tabular data and non-tabular data; (3) determining the importance of an image feature with respect to a modeling task, (4) explaining the value of a modeling target based at least in part on an image feature, and (5) detecting drift in image data. In some cases, multi-stage models may be developed, wherein a pre-trained feature extraction model extracts low-, mid-, high-, and/or highest-level features of non-tabular data, and a data analytics models uses those features (or features derived therefrom) to perform a data analytics task.

Abstract: Systems and methods of epidemiological modeling using machine learning are provided, and can include receiving values for an occurrence of the infectious disease during a first time period, generating, from a model trained by a machine learning system, predictions for the occurrence of the infectious disease over a second time period, performing, by a simulator using the predictions, one or more simulations of the occurrence of the infectious disease in one or more geographic regions during one or more time periods subsequent to the second time period, and providing, to a user interface, a first simulation of the one or more simulations performed by the simulator for a first geographic region of the one or more geographic regions during a time period of the one or more time periods.

Abstract: Systems and methods are described for interpreting machine learning model predictions. An example method includes: providing a machine learning model configured to receive a plurality of features as input and provide a prediction as output, wherein the plurality of features includes an engineered feature including a combination of two or more parent features; calculating a Shapley value for each feature in the plurality of features; and allocating a respective portion of the Shapley value for the engineered feature to each of the two or more parent features.

Abstract: Training computer models by generating time-aware training datasets is provided. A system receives a secondary dataset to be combined with a primary dataset for generation of a training dataset. The primary dataset includes a plurality of data records where at least one data record corresponds to a time-of-prediction value corresponding to a timestamp at which at least one data record was used to generate a prediction. The secondary dataset includes a plurality of features where at least one feature corresponds to a timestamp value. The system selects a feature within the secondary dataset with a timestamp that precedes or matches a time-of-prediction value for a corresponding data record within the primary dataset. The system generates the training dataset that includes the primary dataset and the selected feature. The system trains a model using the generated training dataset.

Abstract: Systems and methods are described for interpreting machine learning model predictions. An example method includes: providing a machine learning model configured to receive a plurality of features as input and provide a prediction as output, wherein the plurality of features includes an engineered feature including a combination of two or more parent features; calculating a Shapley value for each feature in the plurality of features; and allocating a respective portion of the Shapley value for the engineered feature to each of the two or more parent features.

Abstract: Systems and techniques for predictive data analytics are described. In a method for selecting a predictive model for a prediction problem, the suitabilities of predictive modeling procedures for the prediction problem may be determined based on characteristics of the prediction problem and/or on attributes of the respective modeling procedures. A subset of the predictive modeling procedures may be selected based on the determined suitabilities of the selected modeling procedures for the prediction problem. A resource allocation schedule allocating computational resources for execution of the selected modeling procedures may be generated, based on the determined suitabilities of the selected modeling procedures for the prediction problem. Results of the execution of the selected modeling procedures in accordance with the resource allocation schedule may be obtained. A predictive model for the prediction problem may be selected based on those results.

Abstract: Systems and techniques for predictive data analytics are described. In a method for selecting a predictive model for a prediction problem, the suitabilities of predictive modeling procedures for the prediction problem may be determined based on characteristics of the prediction problem and/or on attributes of the respective modeling procedures. A subset of the predictive modeling procedures may be selected based on the determined suitabilities of the selected modeling procedures for the prediction problem. A resource allocation schedule allocating computational resources for execution of the selected modeling procedures may be generated, based on the determined suitabilities of the selected modeling procedures for the prediction problem. Results of the execution of the selected modeling procedures in accordance with the resource allocation schedule may be obtained. A predictive model for the prediction problem may be selected based on those results.

Abstract: Methods for detection of anomalous data samples from a plurality of data samples are provided. In some embodiments, an anomaly detection procedure that includes a plurality of tasks is executed to identify the anomalous data samples from the plurality of data samples.

Abstract: A predictive modeling method may include determining a time interval of time-series data; identifying one or more variables of the data as targets; determining a forecast range and a skip range associated with a prediction problem represented by the data; generating training data and testing data from the time-series data; fitting a predictive model to the training data; and testing the fitted model on the testing data. The forecast range may indicate a duration of a period for which values of the targets are to be predicted. The skip range may indicate a temporal lag between the time period corresponding to the data used to make predictions and the time period corresponding to the predictions. The skip range may separate input data subsets representing model inputs from subsets representing model outputs, and separate test data subsets representing model inputs from subsets representing validation data.

Abstract: A predictive modeling method may include obtaining a fitted, first-order predictive model configured to predict values of output variables based on values of first input variables; and performing a second-order modeling procedure on the fitted, first-order model, which may include: generating input data including observations including observed values of second input variables and predicted values of the output variables; generating training data and testing data from the input data; generating a fitted second-order model of the fitted first-order model by fitting a second-order model to the training data; and testing the fitted, second-order model of the first-order model on the testing data. Each observation of the input data may be generated by (1) obtaining observed values of the second input variables, and (2) applying the first-order predictive model to corresponding observed values of the first input variables to generate the predicted values of the output variables.

Abstract: A method for determining the predictive value of a feature may include: (a) performing predictive modeling procedures associated with respective predictive models, wherein performing each modeling procedure includes fitting the associated model to an initial dataset representing an initial prediction problem; (b) determining a first accuracy score of each of the fitted models, representing an accuracy with which the fitted model predicts an outcome of the initial prediction problem; (c) shuffling values of a feature across observations included in the initial dataset, thereby generating a modified dataset representing a modified prediction problem; (d) determining a second accuracy score of each of the fitted models, representing an accuracy with which the fitted model predicts an outcome of the modified prediction problem; and (e) determining a model-specific predictive value of the feature for each of the fitted models based on the first and second accuracy scores of the fitted model.

Abstract: A predictive modeling method may include obtaining a fitted, first-order predictive model configured to predict values of output variables based on values of first input variables; and performing a second-order modeling procedure on the fitted, first-order model, which may include: generating input data including observations including observed values of second input variables and predicted values of the output variables; generating training data and testing data from the input data; generating a fitted second-order model of the fitted first-order model by fitting a second-order model to the training data; and testing the fitted, second-order model of the first-order model on the testing data. Each observation of the input data may be generated by (1) obtaining observed values of the second input variables, and (2) applying the first-order predictive model to corresponding observed values of the first input variables to generate the predicted values of the output variables.

Abstract: A predictive modeling method may include determining a time interval of time-series data; identifying one or more variables of the data as targets; determining a forecast range and a skip range associated with a prediction problem represented by the data; generating training data and testing data from the time-series data; fitting a predictive model to the training data; and testing the fitted model on the testing data. The forecast range may indicate a duration of a period for which values of the targets are to be predicted. The skip range may indicate a temporal lag between the time period corresponding to the data used to make predictions and the time period corresponding to the predictions. The skip range may separate input data subsets representing model inputs from subsets representing model outputs, and separate test data subsets representing model inputs from subsets representing validation data.

Abstract: A method for determining the predictive value of a feature may include: (a) performing predictive modeling procedures associated with respective predictive models, wherein performing each modeling procedure includes fitting the associated model to an initial dataset representing an initial prediction problem; (b) determining a first accuracy score of each of the fitted models, representing an accuracy with which the fitted model predicts an outcome of the initial prediction problem; (c) shuffling values of a feature across observations included in the initial dataset, thereby generating a modified dataset representing a modified prediction problem; (d) determining a second accuracy score of each of the fitted models, representing an accuracy with which the fitted model predicts an outcome of the modified prediction problem; and (e) determining a model-specific predictive value of the feature for each of the fitted models based on the first and second accuracy scores of the fitted model.

Abstract: A predictive modeling method may include obtaining a fitted, first-order predictive model configured to predict values of output variables based on values of first input variables; and performing a second-order modeling procedure on the fitted, first-order model, which may include: generating input data including observations including observed values of second input variables and predicted values of the output variables; generating training data and testing data from the input data; generating a fitted second-order model of the fitted first-order model by fitting a second-order model to the training data; and testing the fitted, second-order model of the first-order model on the testing data. Each observation of the input data may be generated by (1) obtaining observed values of the second input variables, and (2) applying the first-order predictive model to corresponding observed values of the first input variables to generate the predicted values of the output variables.

Abstract: A method for determining the predictive value of a feature may include: (a) performing predictive modeling procedures associated with respective predictive models, wherein performing each modeling procedure includes fitting the associated model to an initial dataset representing an initial prediction problem; (b) determining a first accuracy score of each of the fitted models, representing an accuracy with which the fitted model predicts an outcome of the initial prediction problem; (c) shuffling values of a feature across observations included in the initial dataset, thereby generating a modified dataset representing a modified prediction problem; (d) determining a second accuracy score of each of the fitted models, representing an accuracy with which the fitted model predicts an outcome of the modified prediction problem; and (e) determining a model-specific predictive value of the feature for each of the fitted models based on the first and second accuracy scores of the fitted model.

Abstract: A predictive modeling method may include determining a time interval of time-series data; identifying one or more variables of the data as targets; determining a forecast range and a skip range associated with a prediction problem represented by the data; generating training data and testing data from the time-series data; fitting a predictive model to the training data; and testing the fitted model on the testing data. The forecast range may indicate a duration of a period for which values of the targets are to be predicted. The skip range may indicate a temporal lag between the time period corresponding to the data used to make predictions and the time period corresponding to the predictions. The skip range may separate input data subsets representing model inputs from subsets representing model outputs, and separate test data subsets representing model inputs from subsets representing validation data.

Abstract: Systems and techniques for predictive data analytics are described. In a method for selecting a predictive model for a prediction problem, the suitabilities of predictive modeling procedures for the prediction problem may be determined based on characteristics of the prediction problem and/or on attributes of the respective modeling procedures. A subset of the predictive modeling procedures may be selected based on the determined suitabilities of the selected modeling procedures for the prediction problem. A resource allocation schedule allocating computational resources for execution of the selected modeling procedures may be generated, based on the determined suitabilities of the selected modeling procedures for the prediction problem. Results of the execution of the selected modeling procedures in accordance with the resource allocation schedule may be obtained. A predictive model for the prediction problem may be selected based on those results.

Abstract: Systems and techniques for predictive data analytics are described. In a method for selecting a predictive model for a prediction problem, the suitabilities of predictive modeling procedures for the prediction problem may be determined based on characteristics of the prediction problem and/or on attributes of the respective modeling procedures. A subset of the predictive modeling procedures may be selected based on the determined suitabilities of the selected modeling procedures for the prediction problem. A resource allocation schedule allocating computational resources for execution of the selected modeling procedures may be generated, based on the determined suitabilities of the selected modeling procedures for the prediction problem. Results of the execution of the selected modeling procedures in accordance with the resource allocation schedule may be obtained. A predictive model for the prediction problem may be selected based on those results.