Abstract: Systems and methods are provided by which a machine learning model may be executed to determine the probability that a given user will respond correctly to a given assessment item of a digital assessment on their first attempt. The machine learning model may process feature data corresponding to the user and the assessment item in order to determine the probability. The feature data may be calculated periodically and/or in real time or near-real time according to a machine learning model definition based on assessment data corresponding to the user's activity and/or based on responses submitted by all users to the assessment item and/or to content related to the assessment item.

Abstract: Systems and methods are described which relate to machine learning model validation. A first machine learning model may be trained to dependent variable data for a first population. A second machine learning model may be trained to simulate dependent variable data for the first population. The second machine learning model may then be applied to student activity data of a second population having different characteristics from the first population to produce simulated dependent variable data. The first machine learning model may then generate predictions for the second population, which may be validated via comparison to the simulated dependent variable data. A given simulated dependent variable value may be generated by the second machine learning model at a specific time TX, where some features input to the machine learning model may be derived from datapoints occurring before TX and others being derived from datapoints occurring after TX.

Abstract: Systems and methods are provided by which an adaptive learning engine may be executed to determine the probability that a given user will respond correctly to a given assessment item of a digital assessment on their first attempt. The adaptive learning engine may apply one or more machine learning models to feature data corresponding to the user and the assessment item in order to determine the probability. The feature data may be calculated periodically and/or in real time or near-real time according to a machine learning model definition based on assessment data corresponding to the user's activity and/or based on responses submitted globally by users to the assessment item and/or to content related to the assessment item. Based on the correct first attempt probability, the adaptive learning engine may identify and recommend assessment items for which a user should be preemptively assigned credit.

Abstract: Systems and methods are described which relate to machine learning model validation. A first machine learning model may be trained to dependent variable data for a first population. A second machine learning model may be trained to simulate dependent variable data for the first population. The second machine learning model may then be applied to student activity data of a second population having different characteristics from the first population to produce simulated dependent variable data. The first machine learning model may then generate predictions for the second population, which may be validated via comparison to the simulated dependent variable data. A given simulated dependent variable value may be generated by the second machine learning model at a specific time TX, where some features input to the machine learning model may be derived from datapoints occurring before TX and others being derived from datapoints occurring after TX.

Abstract: Systems and methods are disclosed related to the identification of key features among features input to a complex predictive model. Logistic models may be created for each of a number of defined clusters of training data used to train the complex predictive model. Coefficients of each logistic model may be analyzed to identify key features that contribute to predictions made by the logistic models. Performance of the logistic models may be compared to that of the complex model to validate the logistic models. When a prediction is made for a given student by the complex predictive model, the student may be assigned to a cluster/by identifying the cluster center having the shortest Euclidean distance to the feature data associated with the student. Key features associated with the assigned cluster may be used as a basis for generating a recommendation for the reducing a risk level of the student.

Abstract: Systems and methods are provided by which an adaptive learning engine may select a machine learning model service to determine a probability that a user will respond correctly to a given assessment item of a digital assessment on their first attempt. The adaptive learning engine may receive a request identifying the user, the assessment item, and request data. A model selector may generate a model reference corresponding to a model definition based on the request data. The feature data to be retrieved and/or calculated may be defined by the model definition. The feature data may be processed by a model service executing a machine learning model selected by the adaptive learning engine based on the model definition. Based on the probability output by the model, the adaptive learning engine may whether the user should be preemptively assigned credit for the assessment item.

Abstract: Systems and methods are provided by which a machine learning model may be executed to determine the probability that a given user will respond correctly to a given assessment item of a digital assessment on their first attempt. The machine learning model may process feature data corresponding to the user and the assessment item in order to determine the probability. The feature data may be calculated periodically and/or in real time or near-real time according to a machine learning model definition based on assessment data corresponding to the user's activity and/or based on responses submitted by all users to the assessment item and/or to content related to the assessment item.

Abstract: Systems and methods are provided by which an adaptive learning engine may be executed to determine the probability that a given user will respond correctly to a given assessment item of a digital assessment on their first attempt. The adaptive learning engine may apply one or more machine learning models to feature data corresponding to the user and the assessment item in order to determine the probability. The feature data may be calculated periodically and/or in real time or near-real time according to a machine learning model definition based on assessment data corresponding to the user's activity and/or based on responses submitted globally by users to the assessment item and/or to content related to the assessment item. Based on the correct first attempt probability, the adaptive learning engine may identify and recommend assessment items for which a user should be preemptively assigned credit.