Abstract: Systems and methods are described which relate to machine learning model validation. A first machine learning model may be trained to dependent variable data for a first population. A second machine learning model may be trained to simulate dependent variable data for the first population. The second machine learning model may then be applied to student activity data of a second population having different characteristics from the first population to produce simulated dependent variable data. The first machine learning model may then generate predictions for the second population, which may be validated via comparison to the simulated dependent variable data. A given simulated dependent variable value may be generated by the second machine learning model at a specific time TX, where some features input to the machine learning model may be derived from datapoints occurring before TX and others being derived from datapoints occurring after TX.

Abstract: Systems and methods are described which relate to machine learning model validation. A first machine learning model may be trained to dependent variable data for a first population. A second machine learning model may be trained to simulate dependent variable data for the first population. The second machine learning model may then be applied to student activity data of a second population having different characteristics from the first population to produce simulated dependent variable data. The first machine learning model may then generate predictions for the second population, which may be validated via comparison to the simulated dependent variable data. A given simulated dependent variable value may be generated by the second machine learning model at a specific time TX, where some features input to the machine learning model may be derived from datapoints occurring before TX and others being derived from datapoints occurring after TX.

Abstract: Systems and methods are disclosed related to the identification of key features among features input to a complex predictive model. Logistic models may be created for each of a number of defined clusters of training data used to train the complex predictive model. Coefficients of each logistic model may be analyzed to identify key features that contribute to predictions made by the logistic models. Performance of the logistic models may be compared to that of the complex model to validate the logistic models. When a prediction is made for a given student by the complex predictive model, the student may be assigned to a cluster/by identifying the cluster center having the shortest Euclidean distance to the feature data associated with the student. Key features associated with the assigned cluster may be used as a basis for generating a recommendation for the reducing a risk level of the student.