Abstract: Resource provisioning systems and methods are described. In an embodiment, a system includes a plurality of shared storage devices collectively storing database data, an execution platform, and a compute service manager. The compute service manager is configured to determine a task to be executed in response to a trigger event and determine a query plan for executing the task, wherein the query plan comprises a plurality of discrete subtasks. The compute service manager is further configured to assign the plurality of discrete subtasks to one or more nodes of a plurality of nodes of the execution platform, determine whether execution of the task is complete, and in response to determining the execution of the task is complete, store a record in the plurality of shared storage devices indicating the task was completed.

Abstract: Systems, methods, and devices for generating a materialized view over an external table. A method includes connecting a database platform to an external table such that the database platform has read access for the external table and does not have write access for the external table. The method includes generating, by the database platform, a materialized view over the external table. The method includes receiving a notification that a modification has been made to the external table, the modification comprising one or more of an addition of a file, a deletion of a file, or an update to a file in a source location for the external table. The method includes, in response to the external table being modified, refreshing the materialized view such that the materialized view comprises an accurate representation of the external table.

Abstract: Resource provisioning systems and methods are described. In an embodiment, a system includes a plurality of shared storage devices collectively storing database data, an execution platform, and a compute service manager. The compute service manager is configured to determine a task to be executed in response to a trigger event and determine a query plan for executing the task, wherein the query plan comprises a plurality of discrete subtasks. The compute service manager is further configured to assign the plurality of discrete subtasks to one or more nodes of a plurality of nodes of the execution platform, determine whether execution of the task is complete, and in response to determining the execution of the task is complete, store a record in the plurality of shared storage devices indicating the task was completed.

Abstract: A method for sharing data in a multi-tenant database includes receiving, by a target account of a multiple tenant database, access rights of a share object in a first account of the multiple tenant database, wherein the share object having access rights to a database object of the first account and wherein access to the database object of the first account by the target account is based on the access rights of the share object. The method also includes receiving, by one or more processors of the target account, access rights to an alias object, wherein the alias object references the database object of the first account.

Abstract: Example resource management systems and methods are described. In one implementation, a resource manager is configured to manage data processing tasks associated with multiple data elements. An execution platform is coupled to the resource manager and includes multiple execution nodes configured to store data retrieved from multiple remote storage devices. Each execution node includes a cache and a processor, where the cache and processor are independent of the remote storage devices. A metadata manager is configured to access metadata associated with at least a portion of the multiple data elements.

Abstract: A method, apparatus, and system for join operations of a plurality of relations that are distributed over a plurality of storage locations over a network of computing components.

Abstract: Joining data using a disjunctive operator using a lookup table is described. An example computer-implemented method can include receiving a query with a set of conjunctive predicates and a set of disjunctive predicates. The method may also include generating a lookup table for each predicate in the sets of conjunctive predicates and disjunctive predicates. The method, for each row in a probe-side table, may also further include looking up a value associated with that row in each of the lookup tables and adding the row to a results set when there is a match. Additionally, the method may also include returning the results set.

Abstract: Joining data using a disjunctive operator is described. An example computer-implemented method can include receiving a query that includes a first disjunctive predicate involving a first table and a second table. The method may also include determining a first set of rows from the first table and generating a filter from the first set of rows. The method may also further include applying the filter to the second table to generate a second set of rows. Additionally, the method may also include joining the first set of rows and the second set of rows using a first disjunctive operator of the first disjunctive predicate to generate a first results set.

Abstract: A shared database platform implements dynamic masking on data shared between users where specific data is masked, transformed, or otherwise modified based on preconfigured functions that are associated with user roles. The shared database platform can implement the masking at runtime dynamically in response to users requesting access to a database object that is associated with one or more masking policies.

Abstract: A method, apparatus, and system for join operations of a plurality of relations that are distributed over a plurality of storage locations over a network of computing components.

Abstract: Systems, methods, and devices for generating a secure join of database data are disclosed. A method includes creating a secure view of datapoints of a consumer account and processing, using a secure user defined function (UDF), the datapoints of the consumer account and datapoints of a provider account to generate a secure join key. The secure join key comprises a hash string of the datapoints of the provider account and the datapoints of the consumer account, and wherein the datapoints of the consumer account are processed via the secure view. The method further includes executing a consumer function to analyze the hash string of datapoints of the secure join key for common data points.

Abstract: The subject technology defines a materialized view over a source table that is associated with a first account of a database. The subject technology defines cross-account access rights to the materialized view for a second account. The subject technology modifies the source table for the materialized view. The subject technology identifies that the materialized view is stale with respect to the source table by merging the materialized view and the source table. The subject technology causes the materialized view to be refreshed with respect to the source table.

Abstract: Systems, methods, and devices for generating and updating cross-account materialized views in multiple tenant database systems. A method includes defining a share object in a first account wherein the share object includes data associated with the first account. The method includes granting cross-account access rights to the share object to a second account such that the second account has access to the share object without copying the share object. The method includes generating a materialized view over the share object. The method includes updating the data associated with the first account. The method includes identifying whether the materialized view is stale with respect to the share object by merging the materialized view and the share object.

Abstract: A query directed at a source table organized into a set of batch units is received. The query includes a pattern matching predicate that specifies a search pattern. A set of N-grams are generated based on the search pattern. A pruning index associated with the source table is accessed. The pruning index comprises a set of filters that index distinct N-grams in each column of the source table. The pruning index is used to identify a subset of batch units to scan for matching data based on the set of N-grams generated for the search pattern. The query is processed by scanning the subset of batch units.

Abstract: A system and method for managing data storage and data access with querying data in a distributed system without buffering the results on intermediate operations in disk storage.

Abstract: Example resource provisioning systems and methods are described. In one implementation, an execution platform accesses multiple remote storage devices. The execution platform includes multiple virtual warehouses, each of which includes a cache to store data retrieved from the remote storage devices and a processor that is independent of the remote storage devices. A resource manager is coupled to the execution platform and monitors received data processing requests and resource utilization. The resource manager also determines whether additional virtual warehouses are needed based on the data processing requests and the resource utilization. If additional virtual warehouses are needed, the resource manager provisions a new virtual warehouse.

Abstract: The subject technology creates partitions based on changes to a table, at least one of the one or more partitions overlapping with respect to values of one or more attributes with at least one of another partition and a previous partition. The subject technology maintains states for the partitions, each state from the plurality of states representing a particular degree of clustering of the table. The subject technology determines a number of overlapping partitions and a depth of the overlapping partitions, and determines a clustering ratio based at least in part on the number of overlapping partitions and the depth. The subject technology reclusters partitions of the table to increase the clustering ratio, the clustering ratio determined by at least a proportion of rows in a layout of the table that satisfy an ordering criteria based at least in part a particular attribute of the one or more attributes.

Abstract: The subject technology receives a query plan, the query plan comprising a set of query operations, the set of query operations including at least one aggregation. The subject technology analyzes the at least one aggregation to generate a modified query plan, the modified query plan including at least a top aggregation operator, an intermediate aggregation operator, and a bottom aggregation operator. The subject technology performs, with respect to the intermediate aggregation operator, at least one operation comprising: the subject technology receives an input intermediate data type; the subject technology performs an internalize operation on the input intermediate data type to generate an internal state; the subject technology performs an accumulate operation on the internal state to generate intermediate data; and the subject technology performs an externalize operation on the intermediate data to generate an output data type.

Abstract: The subject technology creates one or more partitions based on changes to a table, at least one of the one or more partitions overlapping with respect to values of one or more attributes with at least one of another partition and a previous partition, the creating of the one or more partitions resulting in a decrease in a degree of clustering of the table. The subject technology determines, after creating the one or more partitions, that the degree of clustering of the table is below a clustering threshold, the clustering threshold corresponding to a clustering ratio, the clustering ratio determined by at least a proportion of rows in a layout of the table that satisfy an ordering criteria based at least in part a particular attribute of the one or more attributes. The subject technology increases the degree of clustering of the table by reclustering partitions of the table.

Abstract: The subject technology generates a materialized view over a share object, the share object including data associated with a first account. The subject technology merges the materialized view and the share object. The subject technology determines that the materialized view is stale with respect to the share object based at least in part on the merging. The subject technology generates a refreshed materialized view with respect to the share object in response to the determining. The subject technology shares the refreshed materialized view to a second account in response to a request from the second account.