Abstract: In order to efficiently present data extracted from a captured image of an assessment document, a single bitmapped composite image is constructed from multiple images or components such that each layer or content type is stored within discrete bits within each pixel. Different layers are then shown, hidden, emphasized, or otherwise manipulated within a display process through the use of palette color manipulation. Applications to other than assessment documents are discussed.

Abstract: A system and method for automated assessment of constrained constructed responses provides for automatic, e.g., programmatic, assessing of a test subject response to a constrained constructed response item or question. In one embodiment, paper test materials with test subject markings thereon are converted to electronic form. Test subject markings are further isolated, for example, by registration, sizing and removal of markings other than a graphic response. The subject markings are further compared with a baseline response, correctness deviation criteria and incorrectness deviation criteria (e.g., correctness, incorrectness, substantial correctness, substantial incorrectness, and so on) from which one or more of a score, further learning or other assessment results may be determined. Cluster analysis or other post-evaluation or post-assessment processing may also be conducted, or re-evaluation or re-assessment may also be conducted.

Abstract: Methods and apparatus are disclosed for locating an area of interest within a digital image of a form captured by an imaging scanner. Specific examples include methods and apparatus for optical mark reading with a digital imaging scanner. In many of the methods, an image of a response form is captured by a scanner, and “target” areas for possible responses are located within the image based upon an expected location being adjusted as necessary for certain error-inducing defects in the forms or scanning process. Also disclosed are steps to normalize the darkness values of pixels captured from an optically scanned form.

Abstract: In a method and system to extract data from handwritten information when the information is captured as sequences of strokes, vectors, or marks, temporal data is stored within the color or gray-scale encoding of pixel values within a bitmapped image. A bitmapped image is used to create a derived image suitable for data extraction and a method of analyzing the image, including the encoded temporal data, to yield an adjusted image that better reflects the intent of the person who created the handwriting. While the primary application of the invention is in processing handwritten responses to assessment items, the system and method can be advantageously used to extract data from or to improve the presentation of information from an electronic representation of temporal human marks in many applications. Additionally, the system and method can be advantageously used to store and process other types of data within a bitmapped image.

Abstract: Methods and apparatus are disclosed for locating an area of interest within a digital image of a form captured by an imaging scanner. Specific examples include methods and apparatus for optical mark reading with a digital imaging scanner. In many of the methods, an image of a response form is captured by a scanner, and target areas for possible responses are located within the image based upon an expected location being adjusted as necessary for certain error-inducing defects in the forms or scanning process. Also disclosed are steps to normalize the darkness values of pixels captured from an optically scanned form.

Abstract: In order to efficiently present data extracted from a captured image of an assessment document, a single bitmapped composite image is constructed from multiple images or components such that each layer or content type is stored within discrete bits within each pixel. Different layers are then shown, hidden, emphasized, or otherwise manipulated within a display process through the use of palette color manipulation. Applications to other than assessment documents are discussed.

Abstract: A system and method for reading into memory a plurality of scanned image files, obtaining a set of parameters describing the extent to which each scanned image file deviates from a theoretical image file, wherein the set of parameters includes at least one of a horizontal offset, a vertical offset, a horizontal stretch, a vertical stretch, or skew, manipulating a first scanned image file from the plurality according to the set of parameters for the scanned image file, reading into memory a manipulation control, conditionally manipulating a second scanned image file from the plurality based on a value of the manipulation control, and saving at least one manipulated image.

Abstract: Methods and apparatus are disclosed for locating an area of interest within a digital image of a form captured by an imaging scanner. Specific examples include methods and apparatus for optical mark reading with a digital imaging scanner. In many of the methods, an image of a response form is captured by a scanner, and “target” areas for possible responses are located within the image based upon an expected location being adjusted as necessary for certain error-inducing defects in the forms or scanning process. Also disclosed are steps to normalize the darkness values of pixels captured from an optically scanned form.

Abstract: Embodiments of the invention may be used to capture student markings on assessment documents. Embodiments of the invention may be used for selected response questions or constructed response questions. Embodiments of the invention may be particularly suited to process exceptional marks.

Abstract: Methods and apparatus are disclosed for identifying intended response marks in an optically scanned image of a response sheet having key marks and response targets printed in the same color. Pixel darkness values assigned by a digital imaging scanner are used to locate key marks and response targets and to identify response marks. Areas of the sheet may be selectively filtered to mask the response targets. Alternatively, a color imaging scanner may be used to produce a set of color values, such as RBG values, for each pixel. Software can act as a virtual filter by using the RBG values to detect or to mask a color in the image.

Abstract: Methods are disclosed for distributing digitized electronic representations of representative responses to non-objective assessment tasks and established scores for the responses to readers in order to train the readers to determine scores for responses in accordance with a scoring criteria, to qualify the readers to determine scores for responses in accordance with the scoring criteria, and to monitor how well a reader is determining scores for such responses in accordance with the scoring criteria. In disclosed embodiments, the methods provide such distribution for multiple scorers on a network located at a single site, for multiple scorers on a network remote from a central site; and scoring by individual scorers remote from the central site. Distribution may be accomplished by transferring data from electronic files using diverse techniques such as magnetic storage discs, CD-ROM, cable or high-speed public communications network, in accordance with the needs of the particular job and setting.

Abstract: Methods and apparatus are disclosed for identifying intended response marks in an optically scanned image of a response sheet having key marks and response targets printed in the same color. Pixel darkness values assigned by a digital imaging scanner are used to locate key marks and response targets and to identify response marks. Areas of the sheet may be selectively filtered to mask the response targets. Alternatively, a color imaging scanner may be used to produce a set of color values, such as RBG values, for each pixel. Software can act as a virtual filter by using the RBG values to detect or to mask a color in the image.

Abstract: The current invention discloses methods to expedite and simplify a range finding process with resulting savings in time and cost. The invention utilizes digitized representations of respondent answers instead of documents containing the student responses. The invention uses methods of using performance indicators other than the response itself to identify a targeted subset of individual student responses within the sample which are likely to be exemplars of any given score point. In the preferred implementation of this invention, scanned images or other digitized representations of the student responses, and electronically recorded performance indicators are used so that a computer system can quickly and efficiently identify and present the targeted subset to the expert readers of a range finding team. The invention allows expanding the sample of responses available to range finding, up to the entire test administration.

Abstract: Methods are disclosed for distributing digitized electronic representations of representative responses to non-objective assessment tasks and established scores for the responses to readers in order to train the readers to determine scores for responses in accordance with a scoring criteria, to qualify the readers to determine scores for responses in accordance with the scoring criteria, and to monitor how well a reader is determining scores for such responses in accordance with the scoring criteria. In disclosed embodiments, the methods provide such distribution for multiple scorers on a network located at a single site, for multiple scorers on a network remote from a central site; and scoring by individual scorers remote from the central site. Distribution may be accomplished by transferring data from electronic files using diverse techniques such as magnetic storage discs, CD-ROM, cable or high-speed public communications network, in accordance with the needs of the particular job and setting.

Abstract: Methods and apparatus are shown that provide an improved system for recording intended votes in elections. The system includes the voter verifying the interpretation of a voter's ballot by a ballot counting device, an electronic recording of the voter's intended votes on the ballot for possible recounting, and providing a printed receipt of the votes case by the voter. In the preferred embodiment of the invention, voters record their vote by making marks on paper ballots, which are interpreted by reading device. Alternative implementations are provided using mechanical voting, optical reading, or electronic voting devices including interactive systems such as would be used on the Internet. A recording and receipt device presents the interpreted vote for verification by the voter before recording the choices. This creates a less ambiguous record of the choices. The recording and receipt then present the voter with a receipt showing the votes recorded.

Abstract: Methods are disclosed for distributing digitized electronic representations of representative responses to non-objective assessment tasks and established scores for the responses to readers in order to train the readers to determine scores for responses in accordance with a scoring criteria, to qualify the readers to determine scores for responses in accordance with the scoring criteria, and to monitor how well a reader is determining scores for such responses in accordance with the scoring criteria. In disclosed embodiments, the methods provide such distribution for multiple scorers on a network located at a single site, for multiple scorers on a network remote from a central site; and scoring by individual scorers remote from the central site. Distribution may be accomplished by transferring data from electronic files using diverse techniques such as magnetic storage discs, CD-ROM, cable or high-speed public communications network, in accordance with the needs of the particular job and setting.

Abstract: The current invention discloses methods to expedite and simplify a range finding process with resulting savings in time and cost. The invention utilizes digitized representations of respondent answers instead of documents containing the student responses. The invention uses methods of using performance indicators other than the response itself to identify a targeted subset of individual student responses within the sample which are likely to be exemplars of any given score point. In the preferred implementation of this invention, scanned images or other digitized representations of the student responses, and electronically recorded performance indicators are used so that a computer system can quickly and efficiently identify and present the targeted subset to the expert readers of a range finding team. The invention allows expanding the sample of responses available to range finding, up to the entire test administration.

Abstract: Methods are disclosed for distributing digitized electronic representations of representative responses to non-objective assessment tasks and established scores for the responses to readers in order to train the readers to determine scores for responses in accordance with a scoring criteria, to qualify the readers to determine scores for responses in accordance with the scoring criteria, and to monitor how well a reader is determining scores for such responses in accordance with the scoring criteria. In disclosed embodiments, the methods provide such distribution for multiple scorers on a network located at a single site, for multiple scorers on a network remote from a central site; and scoring by individual scorers remote from the central site. Distribution may be accomplished by transferring data from electronic files using diverse techniques such as magnetic storage discs, CD-ROM, cable or high-speed public communications network, in accordance with the needs of the particular job and setting.

Abstract: Apparatus and methods are disclosed for scoring nonobjective assessment materials by displaying images of the assessment materials or of text in lieu of images on a computer screen, for scoring by a human scorer. A digitized image of the assessment material may be captured by scanning an assessment form and storing the image in an electronic file. The stored digitized image or other data is displayed to the scorer and a score entered by the scorer is stored. In disclosed embodiments, the apparatus provides for scoring by multiple scorers on a network located at a single, central site; scoring by multiple scorers on a network remote from the central site; and scoring by one or more individual scorers remote from the central site. Apparatus and methods are disclosed for controlling and controlling access to assessment materials; for matching the qualifications of the scorers to the assessment materials made available to them for scoring; and for evaluating the performance of the scorers.

Abstract: An apparatus and method are provided for scoring nonobjective assessment materials recorded on paper, such as essays, by displaying digitized images of the assessment materials on computer screens for scoring by human scorers. A digitized image of the assessment material is captured by scanning an assessment form and storing the image in an electronic file. The stored digitized image is displayed to the scorer and a score entered by the scorer is returned to a scoring center where reports are generated. Embodiments of the invention include capturing the image in association with Optical Mark Reading (OMR), capturing identifying information through OMR, training and testing the scorer by displaying images of anchor items and validity items for scoring. A series of computer data files ensure control of the scoring process.