Abstract: Methods and systems are described which use load balancing to efficiently perform queries. Embodiments identify available nodes and divide an index into shards. Embodiments distribute the index across the nodes where each node has at least two distinct shards. Based on a target upper bound number of requests per node, and based on which shards exist at each node, embodiments identify a query routing policy which also includes accounting for node failure. Embodiments route a query based on the identified query routing policy.

Abstract: Methods and systems are described which use load balancing to efficiently perform queries. Embodiments identify available nodes and divide an index into shards. Embodiments distribute the index across the nodes where each node has at least two distinct shards. Based on a target upper bound number of requests per node, and based on which shards exist at each node, embodiments identify a query routing policy which also includes accounting for node failure. Embodiments route a query based on the identified query routing policy.

Abstract: Techniques for compressing a data set in a manner such that the data set is represented as a geometric filter are disclosed. Here, the geometric filter has a small memory footprint, thereby enabling the data set to be indexable without consuming a large amount of memory. One bits can be represented within a bit vector portion of the geometric filter or in an index portion of the geometric filter. Techniques are also provided to determine a level of similarity between two data sets, where the data sets have been compressed such that the data sets are representable as geometric filters. The data sets can be indexed without consuming large amounts of memory, and the similarity determination can be performed rapidly. Approximate minimum spanning trees (MSTs) are also generated to find an optimal compression for the data sets, such that the indexes are smaller and searching operations are faster.

Abstract: Techniques for compressing a data set in a manner such that the data set is represented as a geometric filter are disclosed. Here, the geometric filter has a small memory footprint, thereby enabling the data set to be indexable without consuming a large amount of memory. One bits can be represented within a bit vector portion of the geometric filter or in an index portion of the geometric filter. Techniques are also provided to determine a level of similarity between two data sets, where the data sets have been compressed such that the data sets are representable as geometric filters. The data sets can be indexed without consuming large amounts of memory, and the similarity determination can be performed rapidly. Approximate minimum spanning trees (MSTs) are also generated to find an optimal compression for the data sets, such that the indexes are smaller and searching operations are faster.

Abstract: Techniques for compressing a data set in a manner such that the data set is represented as a geometric filter are disclosed. Here, the geometric filter has a small memory footprint, thereby enabling the data set to be indexable without consuming a large amount of memory. One bits can be represented within a bit vector portion of the geometric filter or in an index portion of the geometric filter. Techniques are also provided to determine a level of similarity between two data sets, where the data sets have been compressed such that the data sets are representable as geometric filters. The data sets can be indexed without consuming large amounts of memory, and the similarity determination can be performed rapidly. Approximate minimum spanning trees (MSTs) are also generated to find an optimal compression for the data sets, such that the indexes are smaller and searching operations are faster.

Abstract: An information retrieval system uses an inverted indexing system composed of hybrid positional posting lists. A hybrid positional posting list includes a number of postings, where each posting contains a single numeric value to represent both the document containing a term and the location of the term in the document. When the number of postings for a document exceeds a threshold, a single-valued posting represents all the locations of a term in a document and the locations within the document. When the number of postings for a document is less than a threshold, each location of a term and the associated document is represented by a single-valued posting. In this manner, the storage requirements for the inverted indexing system are reduced and the processing speed increased.