Abstract: A method includes receiving a database query requesting a database to conditionally return one or more data blocks. The database is stored on memory hardware in communication with the data processing hardware and the database query includes a plurality of parameters characterizing the database query. The method includes generating a set of query plans. Each query plan in the set of query plans is configured to execute the database query using a different order of operations. The method includes training a model using historical database queries and generating, using the trained model, a query plan score for each query plan in the set of query plans. The method includes selecting, using the query plan score of each query plan in the set of query plans, a query plan from the set of query plans. The method also includes executing the database query using the selected query plan.

Abstract: Described is a query optimizer comprising a query tuner that performs actual execution of query fragments to obtain actual results during compilation time, and uses those actual results to select a query plan. The actual results may be combined with estimates for fragments that were not executed. The tree may be traversed in a top-down traversal, processing every node. Alternatively, the tree may be traversed in a bottom-up traversal, re-deriving data for higher nodes as each lower level is completed. A limit, such as a time limit or level limit, may be used to control how much time is taken to determine the execution plan.

Abstract: Histograms formed based on samples of a population, such as histograms created from random page-level samples of a data store, are intelligently scaled to histograms estimating distribution of the entire population of the data store. As an optional optimization, where a threshold number of duplicate samples are observed during page-level sampling, the number of distinct values in the overall population data is presumed to be the number of distinct values in the sample data. Also, during estimation of distinct values of an overall population, a “Chao” estimator can optionally be utilized as a lower bound of the estimate. The resulting estimate is then used when scaling, which can take domain knowledge of the data being scaled into account in order to prevent scaled estimates from exceeding the limits of the domain.

Abstract: Histograms formed based on samples of a population, such as histograms created from random page-level samples of a data store, are intelligently scaled to histograms estimating distribution of the entire population of the data store. As an optional optimization, where a threshold number of duplicate samples are observed during page-level sampling, the number of distinct values in the overall population data is presumed to be the number of distinct values in the sample data. Also, during estimation of distinct values of an overall population, a “Chao” estimator can optionally be utilized as a lower bound of the estimate. The resulting estimate is then used when scaling, which can take domain knowledge of the data being scaled into account in order to prevent scaled estimates from exceeding the limits of the domain.

Abstract: Histograms formed based on samples of a population, such as histograms created from random page-level samples of a data store, are intelligently scaled to histograms estimating distribution of the entire population of the data store. As an optional optimization, where a threshold number of duplicate samples are observed during page-level sampling, the number of distinct values in the overall population data is presumed to be the number of distinct values in the sample data. Also, during estimation of distinct values of an overall population, a “Chao” estimator can optionally be utilized as a lower bound of the estimate. The resulting estimate is then used when scaling, which can take domain knowledge of the data being scaled into account in order to prevent scaled estimates from exceeding the limits of the domain.

Abstract: Described is a query optimizer comprising a query tuner that performs actual execution of query fragments to obtain actual results during compilation time, and uses those actual results to select a query plan. The actual results may be combined with estimates for fragments that were not executed. The tree may be traversed in a top-down traversal, processing every node. Alternatively, the tree may be traversed in a bottom-up traversal, re-deriving data for higher nodes as each lower level is completed. A limit, such as a time limit or level limit, may be used to control how much time is taken to determine the execution plan.

Abstract: A compressed trie has nodes including multiple character sub-strings. Such multiple character storage reduces the number of nodes in the trie, thereby reducing the amount of memory required for storing the trie and reducing the amount of time required to perform matching. Furthermore, in such a compressed trie, sub-strings are stored in a single character string. Each node references its corresponding sub-string by the sub-string's starting position and length in the character string. Multiple nodes may reference a single sub-string. Thus, referencing rather than storing sub-strings in corresponding nodes eliminates repetitive sub-string storage, thereby reducing the amount of memory required for storing the trie.

Abstract: Histograms formed based on samples of a population, such as histograms created from random page-level samples of a data store, are intelligently scaled to histograms estimating distribution of the entire population of the data store. As an optional optimization, where a threshold number of duplicate samples are observed during page-level sampling, the number of distinct values in the overall population data is presumed to be the number of distinct values in the sample data. Also, during estimation of distinct values of an overall population, a “Chao” estimator can optionally be utilized as a lower bound of the estimate.

Abstract: A compressed trie has nodes including multiple character sub-strings. Such multiple character storage reduces the number of nodes in the trie, thereby reducing the amount of memory required for storing the trie and reducing the amount of time required to perform matching. Furthermore, in such a compressed trie, sub-strings are stored in a single character string. Each node references its corresponding sub-string by the sub-string's starting position and length in the character string. Multiple nodes may reference a single sub-string. Thus, referencing rather than storing sub-strings in corresponding nodes eliminates repetitive sub-string storage, thereby reducing the amount of memory required for storing the trie.

Abstract: A compressed trie has nodes including multiple character sub-strings. Such multiple character storage reduces the number of nodes in the trie, thereby reducing the amount of memory required for storing the trie and reducing the amount of time required to perform matching. Furthermore, in such a compressed trie, sub-strings are stored in a single character string. Each node references its corresponding sub-string by the sub-string's starting position and length in the character string. Multiple nodes may reference a single sub-string. Thus, referencing rather than storing sub-strings in corresponding nodes eliminates repetitive sub-string storage, thereby reducing the amount of memory required for storing the trie.

Abstract: A compressed trie has nodes including multiple character sub-strings. Such multiple character storage reduces the number of nodes in the trie, thereby reducing the amount of memory required for storing the trie and reducing the amount of time required to perform matching. Furthermore, in such a compressed trie, sub-strings are stored in a single character string. Each node references its corresponding sub-string by the sub-string's starting position and length in the character string. Multiple nodes may reference a single sub-string. Thus, referencing rather than storing sub-strings in corresponding nodes eliminates repetitive sub-string storage, thereby reducing the amount of memory required for storing the trie.