Abstract: Described are methods and apparatuses, including computer program products, for automatically managing quality of human document review in a review process. The method includes receiving tagging decisions for multiple documents made by a first reviewer during a first time period and sampling a subset of these documents based on a first confidence level and first confidence interval. The method further includes receiving tagging decisions made by a second reviewer related to the subset of the documents, from which values of multiple quality-control metrics are determined. The method further includes calculating a risk-accuracy value based in part on the values of the quality-control metrics and recommending a second confidence level and a second confidence interval for sampling a second set of documents reviewed by the first reviewer during a second time period.