Abstract: Systems and methods are provided for creating and utilizing search indexes with different properties to provide efficient and flexible means for retrieving and ranking high-dimensional content. A first index and a second index are generated with a different sets of properties, which may include a permutation prefix length and a vector quantization scheme, and are each associated with a computational workload. Further, at least one index structure may be created with a posting list associated with a permutation prefix and a set of references, rather than only a permutation prefix, to balance out posting lists of variable lengths within the index. Another embodiment of the disclosure provides for expansion of search results at search time, rather than during indexing, by retrieving posting lists associated with shorter permutation prefix after an initial list is determined to have insufficient recall.

Abstract: Systems and methods are provided for creating and utilizing search indexes with different properties to provide efficient and flexible means for retrieving and ranking high-dimensional content. A first index and a second index are generated with a different sets of properties, which may include a permutation prefix length and a vector quantization scheme, and are each associated with a computational workload. Further, at least one index structure may be created with a posting list associated with a permutation prefix and a set of references, rather than only a permutation prefix, to balance out posting lists of variable lengths within the index. Another embodiment of the disclosure provides for expansion of search results at search time, rather than during indexing, by retrieving posting lists associated with shorter permutation prefix after an initial list is determined to have insufficient recall.

Abstract: Aspects of the present disclosure include a system comprising a machine-readable storage medium storing at least one program and computer-implemented methods for efficiently storing graph data. Consistent with some embodiments, the method includes using the graph data to construct a tree topology bitmap that represents a topology of a portion of the nodes in a tree data structure representing the graph data. The method further includes creating an archive bit array for the tree data structure that includes elements populated with a binary indicator of whether a corresponding leaf node includes a known entry. The method further includes creating a leaf values array comprising a known value for each leaf node in the tree data structure with a known entry. The method further includes storing, in a memory storage device, the tree topology, the archive bit array, and the leaf values array in association with an initial timestamp.

Abstract: Aspects of the present disclosure include a system comprising a machine-readable storage medium storing at least one program and computer-implemented methods for efficiently storing graph data. Consistent with some embodiments, the method includes using the graph data to construct a tree topology bitmap that represents a topology of a portion of the nodes in a tree data structure representing the graph data. The method further includes creating an archive bit array for the tree data structure that includes elements populated with a binary indicator of whether a corresponding leaf node includes a known entry. The method further includes creating a leaf values array comprising a known value for each leaf node in the tree data structure with a known entry. The method further includes storing, in a memory storage device, the tree topology, the archive bit array, and the leaf values array in association with an initial timestamp.

Abstract: Aspects of the present disclosure include a system comprising a machine-readable storage medium storing at least one program and computer-implemented methods for efficiently storing graph data. Consistent with some embodiments, the method includes using the graph data to construct a tree topology bitmap that represents a topology of a portion of the nodes in a tree data structure representing the graph data. The method further includes creating an archive bit array for the tree data structure that includes elements populated with a binary indicator of whether a corresponding leaf node includes a known entry. The method further includes creating a leaf values array comprising a known value for each leaf node in the tree data structure with a known entry. The method further includes storing, in a memory storage device, the tree topology, the archive bit array, and the leaf values array in association with an initial timestamp.

Abstract: A data management system accesses a set of vectors containing binary values generates a corresponding set of sequentially ordered vector blocks. Each vector contains a set of two or more binary values and a numerical vector identifier. The data management system generates a block index based on each corresponding set of sequentially ordered vector blocks. The block index includes a set of vector block arrays, each corresponding to a respective sequential position and including one vector block from each of the sets of sequentially ordered vector blocks that are in the respective sequential position. The vector blocks in each vector block array being are ordered sequentially based on two or more sequential binary values in each respective vector block. For each vector block array, the data management system combines pairs of sequentially ordered vector blocks containing matching sets of two or more binary values into combined vector blocks.

Abstract: One or more factors of a query and one or more search result candidates are identified. A plurality of decision trees are associated, via a data structure, with one or more leaf invalidation pairs for at least a first value of the one or more factors. The one or more search result candidates are scored based at least in part on the associating of the plurality of decision trees with one or more leaf invalidation pairs for at least the first value of the one or more factors within the data structure.

Abstract: A data management system accesses a set of vectors containing binary values and generates vector blocks comprising binary values from each vector. Each of at least a portion of the vector blocks for each vector contain a set of two or more binary values from the vector. The data management system generates a block index based on the vector blocks. The block index includes a set of vector block arrays, each vector block array corresponding to a position in the vectors and including the binary values of a vector block from each vector. The data management system can identify relevant vectors for a target vector by generating vector blocks from the target vector and querying the block index to identify candidate vectors.

Abstract: Systems and methods are provided for creating and utilizing search indexes with different properties to provide efficient and flexible means for retrieving and ranking high-dimensional content. A first index and a second index are generated with a different sets of properties, which may include a permutation prefix length and a vector quantization scheme, and are each associated with a computational workload. Further, at least one index structure may be created with a posting list associated with a permutation prefix and a set of references, rather than only a permutation prefix, to balance out posting lists of variable lengths within the index. Another embodiment of the disclosure provides for expansion of search results at search time, rather than during indexing, by retrieving posting lists associated with shorter permutation prefix after an initial list is determined to have insufficient recall.

Abstract: A data management system accesses a set of vectors containing binary values generates a corresponding set of sequentially ordered vector blocks. Each vector contains a set of two or more binary values and a numerical vector identifier. The data management system generates a block index based on each corresponding set of sequentially ordered vector blocks. The block index includes a set of vector block arrays, each corresponding to a respective sequential position and including one vector block from each of the sets of sequentially ordered vector blocks that are in the respective sequential position. The vector blocks in each vector block array being are ordered sequentially based on two or more sequential binary values in each respective vector block. For each vector block array, the data management system combines pairs of sequentially ordered vector blocks containing matching sets of two or more binary values into combined vector blocks.

Abstract: Aspects of the present disclosure include a system comprising a machine-readable storage medium storing at least one program and computer-implemented methods for efficiently storing graph data. Consistent with some embodiments, the method includes using the graph data to construct a tree topology bitmap that represents a topology of a portion of the nodes in a tree data structure representing the graph data. The method further includes creating an archive bit array for the tree data structure that includes elements populated with a binary indicator of whether a corresponding leaf node includes a known entry. The method further includes creating a leaf values array comprising a known value for each leaf node in the tree data structure with a known entry. The method further includes storing, in a memory storage device, the tree topology, the archive bit array, and the leaf values array in association with an initial timestamp.