Abstract: A method of efficiently constructing and using an index for searches in large datasets is provided. In this method, a dataset can be divided into a plurality of “buckets”, i.e. contiguous, non-overlapping ranges of values of a chosen attribute of elements of the dataset. These buckets can be used to construct discrete index segments. Static index segments (associated with “static” buckets including the more frequently occurring values) are always computed, whereas dynamic index segments (associated with “dynamic” buckets including the less frequently occurring values) can be generated on an as-needed basis and based on the availability of system resources. Thus, search time and storage area can be advantageously reduced because less of the complete index needs to be generated and used.

Abstract: A system architecture optimized for pattern match applications is provided. This system architecture includes a host computer and a pattern match accelerator (PMA), which in turn includes one or more pattern match units (PMUs) and PMU control logic. The PMU control logic can divide a database, transmitted by the host computer, such that each of the PMUs receives a database portion. Each PMU includes a main memory for storing the database portion and a programmable logic device (PLD). The PLD can perform a search and score operation on its database portion. Advantageously, the PLD can be configured to generate an index of the database portion, and then configured to perform the search and score operation using that index. The PMU control logic can assemble the results of the pattern match application from each of the PMUs.