Abstract: Computer-implemented systems and methods are described herein for detecting unusually frequent exactly matching and nearly matching test responses. A plurality of test responses for each of a plurality of test takers is received. Evidence of a circulated key from the plurality of test responses is identified by detecting two or more test takers with exactly matching test responses for a plurality of questions. The strength of the evidence of the circulated key is analyzed by determining the probability that the two or more test takers would produce exactly matching test responses, wherein the strength of the evidence is inversely related to the probability. Test takers with nearly matching test responses to the circulated key are then identified. All test takers with exactly matching and nearly matching test responses are tagged for further investigation.

Abstract: Systems and methods are described for generating a scoring model for responses. A computer-implemented method of calibrating a scoring model using a processing system for scoring examinee responses includes accessing a plurality of training responses for training the scoring model. The plurality of training responses are analyzed to derive values of multiple features (variables) of the training responses. The scoring model is trained based on the values of the multiple features of the training responses and one or more external measures of proficiency for each individual associated with a training response utilized in the training. The one or more external measures are not derived from the training responses. Based on the training, a weight for each of the multiple features is determined. The scoring model is calibrated to include the weights for at least some of the features for scoring examinee responses.

Abstract: Systems and methods are described for generating a scoring model for responses. A computer-implemented method of calibrating a scoring model using a processing system for scoring examinee responses includes accessing a plurality of training responses for training the scoring model. The plurality of training responses are analyzed to derive values of multiple features (variables) of the training responses. The scoring model is trained based on the values of the multiple features of the training responses and one or more external measures of proficiency for each individual associated with a training response utilized in the training. The one or more external measures are not derived from the training responses. Based on the training, a weight for each of the multiple features is determined. The scoring model is calibrated to include the weights for at least some of the features for scoring examinee responses.