Abstract: Mechanisms for automatically grading a large number of solutions provided by learners in response to an open response mathematical question. Each solution is mapped to a corresponding feature vector based on the mathematical expressions occurring in the solution. The feature vectors are clustered using a conventional clustering method, or alternatively, using a presently-disclosed Bayesian nonparametric clustering method. A representative solution is selected from each solution cluster. An instructor supplies a grade for each of the representative solutions. Grades for the remaining solutions are automatically generated based on their cluster membership and the instructor supplied grades. The Bayesian method may also automatically identify the location of an error in a given solution. The error location may be supplied to the learner as feedback. The error location may also be used to extract information from correct solutions. The extracted information may be supplied to a learner as a solution hint.

Abstract: Mechanisms for automatically grading a large number of solutions provided by learners in response to an open response mathematical question. Each solution is mapped to a corresponding feature vector based on the mathematical expressions occurring in the solution. The feature vectors are clustered using a conventional clustering method, or alternatively, using a presently-disclosed Bayesian nonparametric clustering method. A representative solution is selected from each solution cluster. An instructor supplies a grade for each of the representative solutions. Grades for the remaining solutions are automatically generated based on their cluster membership and the instructor supplied grades. The Bayesian method may also automatically identify the location of an error in a given solution. The error location may be supplied to the learner as feedback. The error location may also be used to extract information from correct solutions.

Abstract: A database of questions is designed to test understanding of a set of concepts. A subset of the questions is selected for administering to one or more learners in a test. One desires for the subset to be small, to minimize testing workload for the learners and grading workload for instructors. However, to preserve the ability to accurately estimate learners' knowledge of the concepts, the questions of the subset should be appropriately chosen and not too small in number. We propose among other things a non-adaptive algorithm and an adaptive algorithm for test size reduction (TeSR) using an extended version of the Sparse Factor Analysis (SPARFA) framework. The SPARFA framework is a framework for modeling learner responses to questions. Our new TeSR algorithms find fast approximate solutions to a combinatorial optimization problem that involves minimizing the uncertainly in assessing a learner's knowledge of the concepts.

Abstract: A method and system for automatically determining an optimal re-training interval for a fault diagnoser based on online monitoring of the performance of a classifier are disclosed. The classifier generates a soft measure of membership in association with a class based on a training data. The output of the classifier can be utilized to assign a label to new data and then the members associated with each class can be clustered into one or more core members and potential outliers. A statistical measure can be utilized to determine if the distribution of the outliers is sufficiently different than the core members after enough outliers have been accumulated. If the outliers are different with respect to the core members, then the diagnoser can be re-trained; otherwise, the output of the classifier can be fed to the fault diagnoser.