Abstract: A computer-implemented method stratified elusion includes selecting hypothetical cutoff ranks when a stopping point is reached, calculating for each respective cutoff rank a recall value, elusion value, and remaining count; and displaying each respective cutoff rank, recall value, elusion value, and remaining count. A stratified elusion system includes a processor and a memory storing instructions that, when executed, cause the system to select cutoff ranks when a stopping point is reached, calculate for each respective cutoff rank a recall value, elusion value, and remaining count; and display each respective cutoff rank, recall value, elusion value, and remaining count.

Abstract: A computer-implemented method, computing system, and non-transitory computer-readable medium are disclosed for configuring a machine learning-assisted review process. The method includes receiving user-defined parameters, retrieving a set of documents based on these parameters, and displaying the documents for user review and coding. Coding decisions are associated with the documents and used to modify training parameters for the machine learning process, which includes employing various neural network models such as recurrent, convolutional, and deep learning neural networks. The system and medium further involve creating, storing, and adjusting machine learning models based on coding decisions. The process aims to enhance document review efficiency by adapting machine learning models to user feedback, ultimately displaying progress and indicating when a review process has reached a predetermined stopping point.

Abstract: A computer-implemented method includes based on a calculated first estimated error rate, second estimated error rate, first uncertain rank count, second uncertain rank count, and target error rate, displaying a stopping point indication. A computing system includes a processor; and a memory storing instructions that, when executed, cause the computing system to: based on a calculated first estimated error rate, second estimated error rate, first uncertain rank count, second uncertain rank count, and target error rate, display a stopping point indication.

Abstract: A computer-implemented method includes based on a calculated first estimated error rate, second estimated error rate, first uncertain rank count, second uncertain rank count, and target error rate, displaying a stopping point indication. A computing system includes a processor; and a memory storing instructions that, when executed, cause the computing system to: based on a calculated first estimated error rate, second estimated error rate, first uncertain rank count, second uncertain rank count, and target error rate, display a stopping point indication. A non-transitory computer readable medium includes program instructions that when executed, cause a computer system to: based on a calculated first estimated error rate, second estimated error rate, first uncertain rank count, second uncertain rank count, and target error rate, display a stopping point indication.

Abstract: A active learning family-based review method includes selecting a document ranked as relevant by a machine learning model, identifying family documents relationally-linked to the ranked relevant document, generating a batch including the ranked relevant document adjacent to the family documents, and displaying the batch in a computing device. An active learning family-based review computing system includes a processor and a memory storing instructions that, when executed, cause the computing system to select a relevant document using machine learning, identify family documents, generate a batch including the relevant document adjacent to the family documents, and display the batch. A non-transitory computer readable medium stores program instructions that when executed, cause a computer system to select a relevant document using machine learning, identify family documents, generate a batch including the relevant document adjacent to the family documents, and display the batch.

Abstract: A computer-implemented method includes based on a calculated first estimated error rate, second estimated error rate, first uncertain rank count, second uncertain rank count, and target error rate, displaying a stopping point indication. A computing system includes a processor; and a memory storing instructions that, when executed, cause the computing system to: based on a calculated first estimated error rate, second estimated error rate, first uncertain rank count, second uncertain rank count, and target error rate, display a stopping point indication. A non-transitory computer readable medium includes program instructions that when executed, cause a computer system to: based on a calculated first estimated error rate, second estimated error rate, first uncertain rank count, second uncertain rank count, and target error rate, display a stopping point indication.

Abstract: A computer-implemented method for identifying a stopping point of an active learning process includes calculating an error rate for a set of documents, determining whether minimum coding exists, checking an error rate, detecting that an uncertainty rate decreases, and displaying an indication that the stopping point has been reached. A computing system for determining a stopping point of an active learning process includes a processors and a memory storing instructions that, when executed, cause the computing system to calculate an error rate, determine that minimum coding exists, check an error rate, detect decreasing uncertainty rate, and display a stopping point indication. A non-transitory computer readable medium storing program instructions that when executed, cause a computer system to calculate an error rate, determine that minimum coding exists, check an error rate, detect decreasing uncertainty rate, and display a stopping point indication.