Abstract: A method for determining the predictive value of a feature may include: (a) performing predictive modeling procedures associated with respective predictive models, wherein performing each modeling procedure includes fitting the associated model to an initial dataset representing an initial prediction problem; (b) determining a first accuracy score of each of the fitted models, representing an accuracy with which the fitted model predicts an outcome of the initial prediction problem; (c) shuffling values of a feature across observations included in the initial dataset, thereby generating a modified dataset representing a modified prediction problem; (d) determining a second accuracy score of each of the fitted models, representing an accuracy with which the fitted model predicts an outcome of the modified prediction problem; and (e) determining a model-specific predictive value of the feature for each of the fitted models based on the first and second accuracy scores of the fitted model.

Abstract: A predictive modeling method may include determining a time interval of time-series data; identifying one or more variables of the data as targets; determining a forecast range and a skip range associated with a prediction problem represented by the data; generating training data and testing data from the time-series data; fitting a predictive model to the training data; and testing the fitted model on the testing data. The forecast range may indicate a duration of a period for which values of the targets are to be predicted. The skip range may indicate a temporal lag between the time period corresponding to the data used to make predictions and the time period corresponding to the predictions. The skip range may separate input data subsets representing model inputs from subsets representing model outputs, and separate test data subsets representing model inputs from subsets representing validation data.

Abstract: A method for developing a generative AI system may include constructing a plurality of generative AI systems, wherein constructing the generative AI systems includes executing at least one modeling blueprint; providing a plurality of queries to each of the generative AI systems, the queries being part of an evaluation dataset; during processing of the queries by each generative AI system, monitoring values of one or more quantitative metrics; providing, for display by a user device, data indicating the values of the quantitative metrics for each generative AI system; and providing, for display by the user device, a recommendation regarding use or non-use of at least one generative AI system included in the plurality of generative AI systems.

Abstract: A predictive modeling method may include obtaining a fitted, first-order predictive model configured to predict values of output variables based on values of first input variables; and performing a second-order modeling procedure on the fitted, first-order model, which may include: generating input data including observations including observed values of second input variables and predicted values of the output variables; generating training data and testing data from the input data; generating a fitted second-order model of the fitted first-order model by fitting a second-order model to the training data; and testing the fitted, second-order model of the first-order model on the testing data. Each observation of the input data may be generated by (1) obtaining observed values of the second input variables, and (2) applying the first-order predictive model to corresponding observed values of the first input variables to generate the predicted values of the output variables.

Abstract: A predictive modeling method may include determining a time interval of time-series data; identifying one or more variables of the data as targets; determining a forecast range and a skip range associated with a prediction problem represented by the data; generating training data and testing data from the time-series data; fitting a predictive model to the training data; and testing the fitted model on the testing data. The forecast range may indicate a duration of a period for which values of the targets are to be predicted. The skip range may indicate a temporal lag between the time period corresponding to the data used to make predictions and the time period corresponding to the predictions. The skip range may separate input data subsets representing model inputs from subsets representing model outputs, and separate test data subsets representing model inputs from subsets representing validation data.

Abstract: A predictive modeling method may include obtaining a fitted, first-order predictive model configured to predict values of output variables based on values of first input variables; and performing a second-order modeling procedure on the fitted, first-order model, which may include: generating input data including observations including observed values of second input variables and predicted values of the output variables; generating training data and testing data from the input data; generating a fitted second-order model of the fitted first-order model by fitting a second-order model to the training data; and testing the fitted, second-order model of the first-order model on the testing data. Each observation of the input data may be generated by (1) obtaining observed values of the second input variables, and (2) applying the first-order predictive model to corresponding observed values of the first input variables to generate the predicted values of the output variables.

Abstract: A method for determining the predictive value of a feature may include: (a) performing predictive modeling procedures associated with respective predictive models, wherein performing each modeling procedure includes fitting the associated model to an initial dataset representing an initial prediction problem; (b) determining a first accuracy score of each of the fitted models, representing an accuracy with which the fitted model predicts an outcome of the initial prediction problem; (c) shuffling values of a feature across observations included in the initial dataset, thereby generating a modified dataset representing a modified prediction problem; (d) determining a second accuracy score of each of the fitted models, representing an accuracy with which the fitted model predicts an outcome of the modified prediction problem; and (e) determining a model-specific predictive value of the feature for each of the fitted models based on the first and second accuracy scores of the fitted model.

Abstract: A predictive modeling method may include obtaining a fitted, first-order predictive model configured to predict values of output variables based on values of first input variables; and performing a second-order modeling procedure on the fitted, first-order model, which may include: generating input data including observations including observed values of second input variables and predicted values of the output variables; generating training data and testing data from the input data; generating a fitted second-order model of the fitted first-order model by fitting a second-order model to the training data; and testing the fitted, second-order model of the first-order model on the testing data. Each observation of the input data may be generated by (1) obtaining observed values of the second input variables, and (2) applying the first-order predictive model to corresponding observed values of the first input variables to generate the predicted values of the output variables.

Abstract: A predictive modeling method may include determining a time interval of time-series data; identifying one or more variables of the data as targets; determining a forecast range and a skip range associated with a prediction problem represented by the data; generating training data and testing data from the time-series data; fitting a predictive model to the training data; and testing the fitted model on the testing data. The forecast range may indicate a duration of a period for which values of the targets are to be predicted. The skip range may indicate a temporal lag between the time period corresponding to the data used to make predictions and the time period corresponding to the predictions. The skip range may separate input data subsets representing model inputs from subsets representing model outputs, and separate test data subsets representing model inputs from subsets representing validation data.

Abstract: A method for determining the predictive value of a feature may include: (a) performing predictive modeling procedures associated with respective predictive models, wherein performing each modeling procedure includes fitting the associated model to an initial dataset representing an initial prediction problem; (b) determining a first accuracy score of each of the fitted models, representing an accuracy with which the fitted model predicts an outcome of the initial prediction problem; (c) shuffling values of a feature across observations included in the initial dataset, thereby generating a modified dataset representing a modified prediction problem; (d) determining a second accuracy score of each of the fitted models, representing an accuracy with which the fitted model predicts an outcome of the modified prediction problem; and (e) determining a model-specific predictive value of the feature for each of the fitted models based on the first and second accuracy scores of the fitted model.