Abstract: A method for a multi-cluster warehouse includes allocating processing units as part of a data warehouse. The processing units access data within one or more databases in cloud storage resources. The method also includes providing one or more queries to each processing unit within the data warehouse. In response to the queries, each processing unit performs database operations on a particular portion of a database table. The method also includes monitoring a workload of the processing units to determine that a processing capacity of the processing units has reached a threshold processing capacity. The method also includes dynamically adding additional processing units to and removing processing units from the data warehouse as needed based on a workload of the processing units.

Abstract: Multiple execution threads process a query directed to a database organized into a plurality of files. In processing the query, a first thread downloads a file from the plurality of files. The file comprises a set of blocks. A parallel scan of the set of blocks is performed by at least the first thread and a second thread to identify data that matches the query. A response to the query is provided based in part on the parallel scan of the set of blocks.

Abstract: Joining data using a disjunctive operator using a lookup table is described. An example computer-implemented method can include receiving a query with a set of conjunctive predicates and a set of disjunctive predicates. The method may also include generating a lookup table for each predicate in the sets of conjunctive predicates and disjunctive predicates. The method, for each row in a probe-side table, may also further include looking up a value associated with that row in each of the lookup tables and adding the row to a results set when there is a match. Additionally, the method may also include returning the results set.

Abstract: Systems and methods for spilling data for hash joins are described. An example method includes determining an amount of available space in a first memory used by a set of relational queries is insufficient for a first relational join query. The first relational join query comprises a join operation. The method also includes determining a set of build memory sizes and a set of probe memory sizes for a set of partitions for the set of relational queries. The method further includes identifying a first partition of the set of partitions based on the set of probe memory sizes and the set of build memory sizes. The method further includes copying the first partition from the first memory to a second memory, wherein the first partition comprises a first build portion and a first probe portion.

Abstract: Joining data using a disjunctive operator is described. An example computer-implemented method can include generating a query plan for a query, wherein there is a join operator expression for each of a plurality of disjunctive predicates and each join operator expression includes at least a conjunctive predicate and a disjunctive operator. The method may also include generating a bloom filter for each of the plurality of disjunctive operators. The method may further include evaluating each of the plurality of join operator expressions using a corresponding one of the plurality of disjunctive operators and bloom filter for each of the plurality of disjunctive predicates to generate a result set.

Abstract: The subject technology provides information, corresponding to properties of a build side of a join operation, to a bloom filter. The subject technology, based at least in part on the information from the bloom filter, determines, during executing of a query plan, at least one property of the join operation to determine whether to switch an aggregation operator to a pass through mode, the at least one property comprising at least a reduction rate. The subject technology, switches, in response to the reduction rate being below a threshold value, the aggregation operator to the pass through mode during runtime of the query plan and, while the aggregation operator is in the pass through mode, an input stream of data goes through the aggregation operator without being analyzed and the input stream of data matches an output stream of data flowing out of the aggregation operator.

Abstract: Joining data using a disjunctive operator is described. An example computer-implemented method can include receiving a query that includes a first disjunctive predicate involving a first table and a second table. The method may also include determining a first set of rows from the first table and generating a filter from the first set of rows. The method may also further include applying the filter to the second table to generate a second set of rows. Additionally, the method may also include joining the first set of rows and the second set of rows using a first disjunctive operator of the first disjunctive predicate to generate a first results set.

Abstract: In one aspect, a computer-implemented method includes detecting, by a server includes one or more processors, a request to perform a hash join operation on a data structure stored in a data storage device, forming a hash lookup dictionary based on lookup results in a hash table, storing the hash lookup dictionary in a cache, and probing, during a probing phase of the hash join operation, the cache.

Abstract: Systems, methods, and devices, for managing data skew during a join operation are disclosed. A method includes computing a hash value for a join operation and detecting data skew on a probe side of the join operation at a runtime of the join operation using a lightweight sketch data structure. The method includes identifying a frequent probe-side join key on the probe side of the join operation during a probe phase of the join operation. The method includes identifying a frequent build-side row having a build-side join key corresponding with the frequent probe-side join key. The method includes asynchronously distributing the frequent build-side row to one or more remote servers.

Abstract: Systems, methods, and devices, for managing data skew during a join operation are disclosed. A method includes computing a hash value for a join operation and detecting data skew on a probe side of the join operation at a runtime of the join operation using a lightweight sketch data structure. The method includes identifying a frequent probe-side join key on the probe side of the join operation during a probe phase of the join operation. The method includes identifying a frequent build-side row having a build-side join key corresponding with the frequent probe-side join key. The method includes asynchronously distributing the frequent build-side row to one or more remote servers.

Abstract: Systems, methods, and devices, for managing data skew during a join operation are disclosed. A method includes computing a hash value for a join operation and detecting data skew on a probe side of the join operation at a runtime of the join operation using a lightweight sketch data structure. The method includes identifying a frequent probe-side join key on the probe side of the join operation during a probe phase of the join operation. The method includes identifying a frequent build-side row having a build-side join key corresponding with the frequent probe-side join key. The method includes asynchronously distributing the frequent build-side row to one or more remote servers.

Abstract: Systems, methods, and devices, for managing data skew during a join operation are disclosed. A method includes computing a hash value for a join operation and detecting data skew on a probe side of the join operation at a runtime of the join operation using a lightweight sketch data structure. The method includes identifying a frequent probe-side join key on the probe side of the join operation during a probe phase of the join operation. The method includes identifying a frequent build-side row having a build-side join key corresponding with the frequent probe-side join key. The method includes asynchronously distributing the frequent build-side row to one or more remote servers.

Abstract: Joining data using a disjunctive operator is described. An example computer-implemented method can include receiving a query that includes a first disjunctive predicate involving a first table and a second table. The method may also include determining a first set of rows from the first table and generating a filter from the first set of rows. The method may also further include applying the filter to the second table to generate a second set of rows. Additionally, the method may also include joining the first set of rows and the second set of rows using a first disjunctive operator of the first disjunctive predicate to generate a first results set.

Abstract: Systems, methods, and devices, for managing data skew during a join operation are disclosed. A method includes computing a hash value for a join operation and detecting data skew on a probe side of the join operation at a runtime of the join operation using a lightweight sketch data structure. The method includes identifying a frequent probe-side join key on the probe side of the join operation during a probe phase of the join operation. The method includes identifying a frequent build-side row having a build-side join key corresponding with the frequent probe-side join key. The method includes asynchronously distributing the frequent build-side row to one or more remote servers.

Abstract: The subject technology receives a query plan, the query plan comprising a set of query operations, the set of query operations including at least one aggregation and a join operation, the join operation including a build side and a probe side. The subject technology inserts an aggregation operator below the probe side of the join operation. The subject technology causes the build side of the join operation to generate a hash table. The subject technology causes the build side of the join operation to generate a bloom filter based at least in part on the hash table and provide information, corresponding to properties of the build side, to a bloom filter. Based at least in part on the information, the subject technology determines at least one property of the join operation to determine whether to switch the aggregation operator to a pass through mode.

Abstract: Joining data using a disjunctive operator is described. An example computer-implemented method can include receiving a query that includes a first disjunctive predicate involving a first table and a second table. The method may also include determining a first set of rows from the first table and generating a filter from the first set of rows. The method may also further include applying the filter to the second table to generate a second set of rows. Additionally, the method may also include joining the first set of rows and the second set of rows using a first disjunctive operator of the first disjunctive predicate to generate a first results set.

Abstract: Joining data using a disjunctive operator using a lookup table is described. An example computer-implemented method can include receiving a query with a set of conjunctive predicates and a set of disjunctive predicates. The method may also include generating a lookup table for each predicate in the sets of conjunctive predicates and disjunctive predicates. The method, for each row in a probe-side table, may also further include looking up a value associated with that row in each of the lookup tables and adding the row to a results set when there is a match. Additionally, the method may also include returning the results set.

Abstract: The subject technology receives a query plan, the query plan comprising a set of query operations, the set of query operations including at least one aggregation. The subject technology analyzes the at least one aggregation to generate a modified query plan, the modified query plan including at least a top aggregation operator, an intermediate aggregation operator, and a bottom aggregation operator. The subject technology performs, with respect to the intermediate aggregation operator, at least one operation comprising: the subject technology receives an input intermediate data type; the subject technology performs an internalize operation on the input intermediate data type to generate an internal state; the subject technology performs an accumulate operation on the internal state to generate intermediate data; and the subject technology performs an externalize operation on the intermediate data to generate an output data type.

Abstract: Data in a micro-partition of a table is stored in a compressed form. In response to a database query on the table comprising a filter, the portion of the data on which the filter operates is decompressed, without decompressing other portions of the data. Using the filter on the decompressed portion of the data, the portions of the data that are responsive to the filter are determined and decompressed. The responsive data is returned in response to the database query. When a query is run on a table that is compressed using dictionary compression, the uncompressed data may be returned along with the dictionary look-up values. The recipient of the data may use the dictionary look-up values for memoization, reducing the amount of computation required to process the returned data.

Abstract: The subject technology provides information, corresponding to properties of a build side of a join operation, to a bloom filter. The subject technology, based at least in part on the information from the bloom filter, determines, during executing of a query plan, at least one property of the join operation to determine whether to switch an aggregation operator to a pass through mode, the at least one property comprising at least a reduction rate. The subject technology, switches, in response to the reduction rate being below a threshold value, the aggregation operator to the pass through mode during runtime of the query plan and, while the aggregation operator is in the pass through mode, an input stream of data goes through the aggregation operator without being analyzed and the input stream of data matches an output stream of data flowing out of the aggregation operator.