Abstract: Disclosed is a virtual rating device that associate online ratings with differential time costs by endowing the graphical user interface that solicited ratings from the users with “physics,” including an initial (default) slider position and friction. When ratings are associated with differential time cost, scores correlated more strongly with objective service quality. The differential time costs optimize information when proportional to the deviation from the median score. Correlations between subjective rating scores and objective service performance can be further improved by boosting time costs for reporting extreme scores. The disclosed device lowers the sample size (and therefore costs) required for obtaining reliable, averaged crowd estimates. The disclosed device improves information quality in online rating and feedback systems. By endowing a rating widget with virtual friction to increase the time cost for reporting extreme scores, one can obtain a more reliable crowd estimates of quality.

Abstract: Disclosed is a method for experimentally optimizing information quality and engagement in crowd sourced evaluation systems. It allows estimating efficiencies of different strategies to provide the greatest opportunity for social learning. It can be used for continuous tuning of social feedback. Also disclosed is a system that allows experimentation with crowd sourced evaluations of virtual services. The experimental system supports large scale online recruitment of participants into web-browser 3D virtual world. The virtual world can be designed to approximate several types of real-world, distributed information systems. Social influences can be experimentally introduced into the feedback system in order to empirically optimize its utility.

Abstract: Disclosed is a virtual rating device that associate online ratings with differential time costs by endowing the graphical user interface that solicited ratings from the users with “physics,” including an initial (default) slider position and friction. When ratings are associated with differential time cost, scores correlated more strongly with objective service quality. The differential time costs optimize information when proportional to the deviation from the median score. Correlations between subjective rating scores and objective service performance can be further improved by boosting time costs for reporting extreme scores. The disclosed device lowers the sample size (and therefore costs) required for obtaining reliable, averaged crowd estimates. The disclosed device improves information quality in online rating and feedback systems. By endowing a rating widget with virtual friction to increase the time cost for reporting extreme scores, one can obtain a more reliable crowd estimates of quality.