Abstract: Systems and methods are provided for processing a group of essays to develop a classifier that detects nonsensical computer-generated essays. A data structure associated with a group of essays is accessed, wherein the group of essays includes nonsensical computer-generated essays and good-faith essays. Both the nonsensical computer-generated essays and the good-faith essays are assigned feature values. The distribution of feature values between the nonsensical computer-generated essays and the good-faith essays is measured. A classifier that detects whether an essay is a nonsensical computer-generated essay is developed, wherein the classifier is developed using the distribution of feature values.

Abstract: Systems and methods are provided for processing a response to essay prompts that request a narrative response. A data structure associated with a narrative essay is accessed. The essay is analyzed to generate an organization subscore, where the organization subscore is generated using a graph metric by identifying content words in each sentence of the essay and populating a data structure with links between related content words in neighboring sentences, wherein the organization subscore is determined based on the links. The essay is analyzed to generate a development subscore, where the development subscore is generated using a transition metric by accessing a transition cue data store and identifying transition words in the essay, wherein the development subscore is based on a number of words in the essay that match words in the transition cue data store. A narrative quality metric is determined based on the organization subscore and the development subscore.

Abstract: Computer-based systems and methods are provided for generating a narrative computer scoring model for assessing story narratives. In one embodiment, supervised machine learning is used to generate the narrative computer scoring model. For example, a collection of training story narratives with assigned scores may be used to train the model. In one embodiment, each training story narrative is processed to extract features that signify content relevance, collocation of commonly used words, coherency, detailing, and expressions of sentiment. These features, as well as others, may be selectively used to train a narrative computer scoring model. Once trained, the model can be used to automatically evaluate story narratives and assign appropriate scores.

Abstract: Systems and methods are provided for automatically scoring essay responses to a prompt using a scoring model. A relevant word corpus and an irrelevant word corpus are accessed. A scoring model is generated by, for each of a plurality of words in the relevant word corpus, determining a topic signature score based on a number of appearances of that word in the relevant word corpus and a number of appearances of that word in the irrelevant word corpus. For each of a plurality of words in an essay response, a topic signature score is determined for that word. A score for the essay response is determined based on the identified topic signature scores.

Abstract: Systems and methods are provided for measuring a user's English language proficiency. A constructed response generated by a user is received, the constructed response being based on a picture. The constructed response is processed to determine a first numerical measure indicative of a presence of one or more grammar errors in the constructed response. The constructed response is processed to determine a second numerical measure indicative of a degree to which the constructed response describes a subject matter of the picture. The constructed response is processed to determine a third numerical measure indicative of a degree of awkward word usage in the constructed response. A model is applied to the first, second, and third numerical measures to determine a score for the constructed response indicative of the user's English language proficiency. The model includes first, second, and third variables with associated first, second, and third weighting factors, respectively.

Abstract: Systems and methods are provided for measuring a user's English language proficiency. A constructed response generated by a user is received, the constructed response being based on a picture. The constructed response is processed to determine a first numerical measure indicative of a presence of one or more grammar errors in the constructed response. The constructed response is processed to determine a second numerical measure indicative of a degree to which the constructed response describes a subject matter of the picture. The constructed response is processed to determine a third numerical measure indicative of a degree of awkward word usage in the constructed response. A model is applied to the first, second, and third numerical measures to determine a score for the constructed response indicative of the user's English language proficiency. The model includes first, second, and third variables with associated first, second, and third weighting factors, respectively.

Abstract: Systems and methods are provided for automatically generating a coherence score for a text using a scoring model. A lexical chain is identified within a text to be scored, where the lexical chain comprises a set of words spaced within the text. A discourse element is identified within the text, where the discourse element comprises a word within the text. A coherence metric is determined based on a relationship between the lexical chain and the discourse element. A coherence score is generated using a scoring model by providing the coherence metric to the scoring model.

Abstract: Systems and methods are provided for detecting punctuation errors in a text including one or more sentences. A sentence including a plurality of words is received, the sentence including one or more preexisting punctuation marks. One or more punctuation marks are determined with a statistical classifier based on a set of rules, to be inserted in the sentence. The determined punctuation marks are compared with the preexisting punctuation marks. A report of punctuation errors is output based on the comparison.

Abstract: Systems and methods are provided for correcting a grammatical error in a text sequence. A first text sequence in a first language is received. The first text sequence is translated to a second language to provide a first translated text. The first text sequence is translated to a third language to provide a second translated text. The third language is different from the second language. The first translated text is translated to the first language to provide a first back translation. The second translated text is translated to the first language to provide a second back translation. A plurality of candidate text sequences that include features of the first back translation and the second back translation are determined. The plurality of candidate text sequences include alternative grammatical options for the first text sequence. The plurality of candidate text sequences are scored with the processing system.

Abstract: In accordance with the teachings described herein, systems and methods are provided for identifying organizational elements in argumentative or persuasive discourse. A text that has been annotated is received. The annotated text includes argumentative or persuasive discourse that includes claims and evidence and organizational elements configured to organize the claims and evidence. Annotations of the annotated text distinguish the organizational elements from the claims and evidence. A rule set or a feature set is identified from the annotated text, where the rule set or the feature set includes textual patterns or word frequency features related to the organizational elements of the annotated text. A model is built based on the annotations and on the rule set or the feature set. The model is configured to identify organizational elements in a new text. The model is applied to the new text.

Abstract: Systems and methods are provided for automatically generating a coherence score for a text using a scoring model. A lexical chain is identified within a text to be scored, where the lexical chain comprises a set of words spaced within the text. A discourse element is identified within the text, where the discourse element comprises a word within the text. A coherence metric is determined based on a relationship between the lexical chain and the discourse element. A coherence score is generated using a scoring model by providing the coherence metric to the scoring model.

Abstract: Systems and methods are provided for measuring a user's English language proficiency. A constructed response generated by a user is received, the constructed response being based on a picture. The constructed response is processed to determine a first numerical measure indicative of a presence of one or more grammar errors in the constructed response. The constructed response is processed to determine a second numerical measure indicative of a degree to which the constructed response describes a subject matter of the picture. The constructed response is processed to determine a third numerical measure indicative of a degree of awkward word usage in the constructed response. A model is applied to the first, second, and third numerical measures to determine a score for the constructed response indicative of the user's English language proficiency. The model includes first, second, and third variables with associated first, second, and third weighting factors, respectively.

Abstract: Systems and methods are provided for correcting a grammatical error in a text sequence. A first text sequence in a first language is received. The first text sequence is translated to a second language to provide a first translated text. The first text sequence is translated to a third language to provide a second translated text. The third language is different from the second language. The first translated text is translated to the first language to provide a first back translation. The second translated text is translated to the first language to provide a second back translation. A plurality of candidate text sequences that include features of the first back translation and the second back translation are determined. The plurality of candidate text sequences include alternative grammatical options for the first text sequence. The plurality of candidate text sequences are scored with the processing system.

Abstract: A concept rater module is utilized to automatically grade or score constructed responses based on a model answer. The concept rater module may be configured to accept a model answer as input. The model answer may be used as a grading key by the concept rater module. The concept rater module may be further configured to accept student responses in a file format. The file format may be ASCII text, a formatted word processing (e.g., WORDPERFECT, MICROSOFT WORD, etc.) and the like. The concept rater module may be further configured to process a student response into a canonical representation of the student response. The canonical representation of the student response is compared against the model answer by the concept rater module. From the comparison, a score is generated which represents that student's ability to cover all the key concepts.

Abstract: An essay is analyzed automatically by accepting the essay and determining whether each of a predetermined set of features is present or absent in each sentence of the essay. For each sentence in the essay a probability that the sentence is a member of a certain discourse element category is calculated. The probability is based on the determinations of whether each feature in the set of features is present or absent. Furthermore, based on the calculated probabilities, a sentence is chosen as the choice for the discourse element category.

Abstract: A concept rater module is utilized to automatically grade or score constructed responses based on a model answer. The concept rater module may be configured to accept a model answer as input. The model answer may be used as a grading key by the concept rater module. The concept rater module may be further configured to accept student responses in a file format. The file format may be ASCII text, a formatted word processing (e.g., WORDPERFECT, MICROSOFT WORD, etc.) and the like. The concept rater module maybe further configured to process a student response into a canonical representation of the student response. The canonical representation of the student response is compared against the model answer by the concept rater module. From the comparison, a score is generated which represents the students ability to cover all the key concepts.