Abstract: Computer-implemented systems and methods are provided for assessing non-native spontaneous speech pronunciation. Speech recognition on digitized speech is performed using a non-native acoustic model trained with non-native speech to generate word hypotheses for the digitized speech. Time alignment is performed between the digitized speech and the word hypotheses using a reference acoustic model trained with native-quality speech. Statistics are calculated regarding individual words and phonemes in the word hypotheses based on the alignment. A plurality of features for use in assessing pronunciation of the speech are calculated based on the statistics, an assessment score is calculated based on one or more of the calculated features, and the assessment score is stored in a computer-readable memory.

Abstract: An essay is analyzed automatically by accepting the essay and determining whether each of a predetermined set of features is present or absent in each sentence of the essay. For each sentence in the essay a probability that the sentence is a member of a certain discourse element category is calculated. The probability is based on the determinations of whether each feature in the set of features is present or absent. Furthermore, based on the calculated probabilities, a sentence is chosen as the choice for the discourse element category.

Abstract: A method and system for determining text coherence in an essay is disclosed. A method of evaluating the coherence of an essay includes receiving an essay having one or more discourse elements and text segments. The one or more discourse elements are annotated either manually or automatically. A text segment vector is generated for each text segment in a discourse element using sparse random indexing vectors. The method or system then identifies one or more essay dimensions and measures the semantic similarity of each text segment based on the essay dimensions. Finally, a coherence level is assigned to the essay based on the measured semantic similarities.

Abstract: A method of determining a mastery level for an examinee from an assessment is disclosed. The method includes receiving one or more of an overall skill level for an examinee, a weight for the overall skill level, a covariate vector for an examinee, and a weight for the covariate vector. An examinee attribute value is computed using one or more of the received values for each examinee and each attribute. The computation of the examinee attribute values can include estimating the value using a Markov Chain Monte Carlo estimation technique. Examinee mastery levels are then assigned based on each examinee attribute level. Dichotomous or polytomous levels can be assigned based on requirements for the assessment.

Abstract: Methods and systems for detecting off-topic essays are described that do not require training using human-scored essays. The methods can detect different types of off-topic essays, such as unexpected topic essays and bad faith essays. Unexpected topic essays are essays that address an incorrect topic. Bad faith essays address no topic. The methods can use content vector analysis to determine the similarity between the essay and one or more prompts. If the essay prompt with which an essay is associated is among the most similar to the essay, the essay is on-topic. Otherwise, the essay is considered to be an unexpected topic essay. Similarly, if the essay is sufficiently dissimilar to all essay prompts, the essay is considered to be a bad faith essay.

Abstract: Systems and methods for detecting collocation errors in a text sample using a reference database from a corpus are provided. Collocation candidates are identified within the text sample based upon syntactic patterns in the text sample. Whether a given collocation candidate contains a collocation error is detected, the detecting including: determining a first association measure using the reference database for the given collocation candidate; determining whether the first association measure satisfies a predetermined condition and identifying the given collocation candidate as proper if the first association measure satisfies the predetermined condition; determining an additional association measure for a variation of the given collocation candidate using the reference database; and determining whether or not the collocation candidate contains an error based upon the additional association measure of the variation.

Abstract: A method of determining a mastery level for an examinee from an assessment is disclosed. The method includes receiving one or more of an overall skill level for an examinee, a weight for the overall skill level, a covariate vector for an examinee, and a weight for the covariate vector. An examinee attribute value is computed using one or more of the received values for each examinee and each attribute. The computation of the examinee attribute values can include estimating the value using a Markov Chain Monte Carlo estimation technique. Examinee mastery levels are then assigned based on each examinee attribute level. Dichotomous or polytomous levels can be assigned based on requirements for the assessment.

Abstract: A method and system for constructing a test using a computer system that performs specification matching during the test creation process is disclosed. A test developer determines one or more test item databases from which to select test items. The test item databases are organized based on psychometric and/or content specifications. The developer can examine the textual passages, artwork or statistical information pertaining to a test item before selecting it by clicking on a designation of the test item in a database. The developer can then add the test item to a list of test items for the test. The test development system updates pre-designated psychometric and content specification information as the developer adds each test item to the test. The test developer can use the specification information to determine whether to add to, subtract from, or modify the list of test items selected for the test.

Abstract: A method and system for performing image mark recognition for a document is disclosed. A document is scanned into a digital image. Reference image marks are sensed from the digital image. The reference image marks may include trigger row marks and/or corner/crop marks. Coordinates denoting the location of cells within the digital image are determined base on the locations of the reference image marks. Response marks are evaluated for darkness, opacity, and/or grayness on a pixel-by-pixel basis. The response marks are each assigned a percentage value based on the ratio between a total color value for a cell and a maximum color value for a cell.

Abstract: A computer-implemented method and system for ensuring quality control over a group of test reports comprising identifying a plurality of areas within a test report that contain data elements, performing optical character recognition on the plurality of areas for each test report in the group of test reports to generate text corresponding to the content in the areas, comparing the content in the areas to corresponding data in a data file from a trusted source, and creating an output report based on the result of the comparison. The test reports may indicate educational test score results and personal identification information for test takers.

Abstract: A method and system for constructing a test using a computer system that performs specification matching during the test creation process is disclosed. A test developer determines one or more test item databases from which to select test items. The test item databases are organized based on psychometric and/or content specifications. The developer can examine the textual passages, artwork or statistical information pertaining to a test item before selecting it by clicking on a designation of the test item in a database. The developer can then add the test item to a list of test items for the test. The test development system updates pre-designated psychometric and content specification information as the developer adds each test item to the test. The test developer can use the specification information to determine whether to add to, subtract from, or modify the list of test items selected for the test.

Abstract: A method and system for encryption and decryption of data files on a web-based computer system includes encrypting the data file in a memory subsystem, such as RAM, storing the encrypted data file in one or more of a plurality of memory locations, analyzing and modifying the encrypted data file, retrieving and decrypting the modified data file, and displaying the decrypted data file on a web browser. The computer system may include one to three servers. The data file may be encrypted and decrypted using any dual-key encryption algorithm, such as PGP, or a single key algorithm. Data transmitted into and out of the computer system may be via an SSL/HTTPS protocol to provide additional security.

Abstract: An on-line teaching and learning system with rapid change-by-change or real time reinforcement signals for students and simultaneous monitoring by the teacher of the actual responses as well as their correctness is disclosed. The system may include at least one teacher computer, a plurality of student computers operably connected to the at least one teacher computer by a communications network. The system may be used in a classroom setting or in a distance-learning environment.

Abstract: An on-line teaching and learning system with rapid change-by-change or real time reinforcement signals for students and simultaneous monitoring by the teacher of the actual responses as well as their correctness is disclosed. The system may include at least one teacher computer, a plurality of student computers operably connected to the at least one teacher computer by a communications network. The system may be used in a classroom setting or in a distance-learning environment.

Abstract: The present disclosure presents a useful metric for assessing the relative difficulty which non-native speakers face in pronouncing a given utterance and a method and systems for using such a metric in the evaluation and assessment of the utterances of non-native speakers. In an embodiment, the metric may be based on both known sources of difficulty for language learners and a corpus-based measure of cross-language sound differences. The method may be applied to speakers who primarily speak a first language speaking utterances in any non-native second language.

Abstract: The present invention provides a method of doing cognitive diagnosis of mental skills, medical and psychiatric diagnosis of diseases and disorders, and in general the diagnosing of latent properties of a set of objects, usually people, for which multiple pieces of binary (dichotomous) information about the objects are available, for example testing examinees using right/wrong scored test questions. Settings where the present invention can be applied but are not limited to include classrooms at all levels, web-based instruction, corporate in-house training, large scale standardized tests, and medical and psychiatric settings. Uses include but are not limited to individual learner feedback, learner remediation, group level educational assessment, and medical and psychiatric treatment.

Abstract: Methods and systems for evaluating the fit of raw data to model data are disclosed. Relationships between an assessment item and a tested attribute, responses from examinees to an assessment item, mastery states for an examinee for a tested attribute, one or more parameters based on expected assessment item performance, estimates for non-tested attributes for each examinee, likelihoods that an examinee that has mastered attributes pertaining to an assessment item will answer the item correctly, likelihoods that an examinee that has not mastered an attribute for an assessment item will answer the item correctly, and/or other variables may be received. For each item and/or for each examinee, a determination of a class for the examinee and/or item may be determined. Statistics may also be generated for each examinee, each item, an examination and/or any other basis.

Abstract: A method and system for determining attribute score levels from an assessment are disclosed. An assessment includes items each testing for at least one attribute. A first distribution is generated having a response propensity represented by a highest level of execution for each attribute tested by the item. An item threshold is determined for at least one score for the first distribution. Each item threshold corresponds to a level of execution corresponding to the score for which the item threshold is determined. For each attribute tested by the item, a second distribution is generated having a response propensity represented by a lowest level of execution for the attribute and the highest level of execution for all other attributes tested by the item. A mean parameter is determined for the second distribution. An attribute score level is determined for the scores based on the item thresholds and the mean parameters.

Abstract: A method and system for automatically generating a scoring model for scoring a speech sample are disclosed. One or more training speech samples are received in response to a prompt. One or more speech features are determined for each of the training speech samples. A scoring model is then generated based on the speech features. At least one of the training speech samples may be a high entropy speech sample. An evaluation speech sample is received and a score is assigned to the evaluation speech sample using the scoring model. The evaluation speech sample may be a high entropy speech sample.

Abstract: A method and system for performing automatic text analysis is described. A local ranking for one or more contexts with respect to a word and a global ranking for one or more contexts are generated. The rankings are based on the frequency with which the contexts appear in a corpus. A statistic may be generated using the local and global rankings, such as a log ratio rank statistic equal to the logarithm of the global rank divided by local rank, to measure the similarity of contexts with respect to words with which they combine. A source matrix of word to context values is then created. Singular value decomposition is used to create sub-matrices from the source matrix. Vectors from the sub-matrices corresponding to context(s) and/or word(s) are then selected to determine term-term or context-context similarity or term-context correspondence.