Abstract: Methods to correct for biases and inaccuracies of subjective data sources are provided. In some instances, data sources provide an indication of quality of an item, and various methods determine an instrinsic quality of the item from the data. In some instances, various methods utilize ratings provided by a collection of raters to determine an intrinsic quality. Biases and inaccuracies of raters can be determined and can be utilized for correction in order to reach an intrinsic quality of an item. A number of applications utilizing quality of an item quality and biases and inaccuracies of raters are also described.