Abstract: The present invention provides systems and methods for statistically analyzing collected data and providing training for the collection of the data. In one embodiment, the system and method are configured to collect data about a plurality of objects and determine a measure of agreement between the collected data and a benchmark. Based on the measure of agreement reliability and a variance of the collected data are determined relative to the benchmark. In training a plurality of raters are instructed to rate at least one of a plurality of objects such that each rater collects data that rates at least one of the objects. A measure of agreement of the data collected by a rater is determined relative to the benchmark, and reliability and variance of the data collected are determined, which are used to evaluate the rater.

Abstract: The present invention provides systems and methods for statistically analyzing collected data and providing training for the collection of the data. In one embodiment, the system and method are configured to collect data about a plurality of objects and determine a measure of agreement between the collected data and a benchmark. Based on the measure of agreement reliability and a variance of the collected data are determined relative to the benchmark. In training a plurality of raters are instructed to rate at least one of a plurality of objects such that each rater collects data that rates at least one of the objects. A measure of agreement of the data collected by a rater is determined relative to the benchmark, and reliability and variance of the data collected are determined, which are used to evaluate the rater.