Abstract: Methods, systems, and articles of manufacture for scalable processing of collections of files are disclosed. These include retrieving a first file processing job from a job list by a first task, decomposing the first file processing job to sub jobs by the first task when a workload associated with the first file processing job exceeds a threshold, listing the sub-jobs in the job list, monitoring of at least one result storage location by the first task for results from execution of respective ones of the sub-jobs by one or more second tasks, detecting completion of the first processing job based upon the monitoring, and outputting a result for the first file processing job based upon the results from execution of respective ones of the sub-jobs. The job list comprising file processing jobs is populated by a main task.

Abstract: Methods, systems, and articles of manufacture for the analysis of large document archives by matching queries against documents are disclosed. These include generating an ordered sequence of query tokens for each query, generating an ordered sequence of document tokens for each document, selecting an ordered sequence of document tokens from the tokenized one or more documents, selecting an ordered sequence of query tokens from the tokenized one or more queries, configuring a buffer to hold a subsequence of the selected ordered sequence of document tokens, comparing the selected ordered sequence of query tokens to successive subsequences of the selected ordered sequence of document tokens in the configured buffer where each of the successive subsequences and the selected ordered sequence of query tokens have the same length in tokens, and determining a match result based upon the comparison.