Abstract: Systems and methods for predictive document coding using continuous active machine learning are described herein. A method uses both a primary queue and a plurality of secondary queues, where each secondary queue is associated with a model for category of documents. The method also repeatedly classifies new batches selected from a large set of documents that have not been reviewed. The classification uses the plurality of models and updates the secondary queues from the best documents in the most recently classified batch. While the method transparently cycles through batches, the most relevant documents are provided to one or more human reviewers from secondary queues via a primary queue. The reviewer confirms relevance or non-relevance in each of the documents for each of the categories. Periodically all the models are retrained using the set of reviewed documents after a selectable number of documents have been reviewed since the most recent retraining.

Abstract: Systems and methods for monitoring the quality of document reviews used in continuous active machine learning are described herein. Two orthogonal processes may be run simultaneously, asynchronously, and continuously. The first process performs continuous active machine learning for training machine classification models. The second process classifies documents that have been reviewed as part of the first process to generate classification scores of the reviewed documents. The original review may be compared to the classification scores using false negative and a false positive thresholds to identify documents that may have been incorrectly reviewed. A master review of identified documents is used to correct original reviews that were incorrect. Original incorrect reviews may be replaced in a training corpus by corrected reviews, and the models may be retrained using the corrected reviews.

Abstract: Systems and methods for improving search results from proprietary search engine technologies and proprietary machine classifiers, without ingesting, copying, or storing, the data to be searched, are described herein. A user sends a query to a proprietary search engines and gets a result document set back. The user may apply a user model for classifying a result document set to generate a result document for review of a user. The reviewed document may be added to a user training corpus, which is then used to retrain the user model. The retrained user model may be applied by the user to generate the next result document for user review and so on until the user model converges to generate relevant documents reliably. Once the user model converges, the user may apply the now reliable user model to generate multiple relevant documents for the user.

Abstract: Systems and methods for monitoring the quality of document reviews used in continuous active machine learning are described herein. Two orthogonal processes may be run simultaneously, asynchronously, and continuously. The first process performs continuous active machine learning for training machine classification models. The second process classifies documents that have been reviewed as part of the first process to generate classification scores of the reviewed documents. The original review may be compared to the classification scores using false negative and a false positive thresholds to identify documents that may have been incorrectly reviewed. A master review of identified documents is used to correct original reviews that were incorrect. Original incorrect reviews may be replaced in a training corpus by corrected reviews, and the models may be retrained using the corrected reviews.

Abstract: Systems and methods for improving search results from proprietary search engine technologies and proprietary machine classifiers, without ingesting, copying, or storing, the data to be searched, are described herein. A user sends a query to a proprietary search engines and gets a result document set back. The user may apply a user model for classifying a result document set to generate a result document for review of a user. The reviewed document may be added to a user training corpus, which is then used to retrain the user model. The retrained user model may be applied by the user to generate the next result document for user review and so on until the user model converges to generate relevant documents reliably. Once the user model converges, the user may apply the now reliable user model to generate multiple relevant documents for the user.

Abstract: Systems and methods for monitoring the quality of document reviews used in continuous active machine learning are described herein. Two orthogonal processes may be run simultaneously, asynchronously, and continuously. The first process performs continuous active machine learning for training machine classification models. The second process classifies documents that have been reviewed as part of the first process to generate classification scores of the reviewed documents. The original review may be compared to the classification scores using false negative and a false positive thresholds to identify documents that may have been incorrectly reviewed. A master review of identified documents is used to correct original reviews that were incorrect. Original incorrect reviews may be replaced in a training corpus by corrected reviews, and the models may be retrained using the corrected reviews.

Abstract: Systems and methods for predictive document coding using continuous active machine learning are described herein. A method uses both a primary queue and a plurality of secondary queues, where each secondary queue is associated with a model for category of documents. The method also repeatedly classifies new batches selected from a large set of documents that have not been reviewed. The classification uses the plurality of models and updates the secondary queues from the best documents in the most recently classified batch. While the method transparently cycles through batches, the most relevant documents are provided to one or more human reviewers from secondary queues via a primary queue. The reviewer confirms relevance or non-relevance in each of the documents for each of the categories. Periodically all the models are retrained using the set of reviewed documents after a selectable number of documents have been reviewed since the most recent retraining.

Abstract: A Purchase Card Maximization apparatus, method, and program product enhances efficiencies realized by an organization through (1) Performance Metrics And Reporting by analyzing applicable purchases through a cost center hierarchical display, measuring performance against baseline targets, highlighting areas in need of improvement, and providing reports that list specific steps to improve purchase card program performance; (2) Usage Compliance by applying specific company policy regulations to current purchase card transactions, record and track violations of company policies by cardholders and organization, automatically providing email notification of violations to cardholders, and providing escalation policy for dealing with repeat offenders; and (3) Audit Implementation And Tracking by applying a specific company policy to determine cardholders to be audited, recording and tracking results of cardholder audits, ensuring due diligence in applying audit strategy, and reporting overall program compliance thro