Abstract: Systems, methods, and devices for querying over an external table are disclosed. A method includes connecting a database platform to an external table such that the database platform has read access for the external table and does not have write access for the external table. The method includes receiving a query comprising a predicate, the query directed at least to data in the external table. The method includes determining, based on metadata, one or more partitions in the external table comprising data satisfying the predicate. The method includes pruning, based on the metadata, all partitions in the external table that do not comprise any data satisfying the predicate. The method includes generating a query plan comprising a plurality of discrete subtasks. The method includes assigning, based on the metadata, the plurality of discrete subtasks to one or more nodes in an execution platform.

Abstract: Systems, methods, and devices for querying over an external table are disclosed. A method includes connecting a database platform to an external table such that the database platform has read access for the external table and does not have write access for the external table. The method includes receiving a query comprising a predicate, the query directed at least to data in the external table. The method includes determining, based on metadata, one or more partitions in the external table comprising data satisfying the predicate. The method includes pruning, based on the metadata, all partitions in the external table that do not comprise any data satisfying the predicate. The method includes generating a query plan comprising a plurality of discrete subtasks. The method includes assigning, based on the metadata, the plurality of discrete subtasks to one or more nodes in an execution platform.

Abstract: Disclosed herein are systems and methods for database replication to a remote deployment with automated fulfillment. In an embodiment, a data platform provisions a remote-deployment account of a data provider at a remote deployment of the data platform with a set of one or more replication-preparation objects associated with a primary-deployment database that is resident in a primary-deployment account of the data provider at a primary deployment of the data platform. After the provisioning, the data platform detects receipt of a database-replication request associated with a data consumer, requesting availability of a local instance of the primary-deployment database in the remote-deployment account of the data provider. The data platform responsively performs one or more remote-deployment-activation operations, which makes available to the data consumer a local instance of the primary-deployment database in the remote-deployment account of the data provider at the remote deployment of the data platform.

Abstract: A system and method for pruning data based on metadata. The method may include receiving a query comprising a plurality of predicates and identifying one or more applicable files comprising database data satisfying at least one of the plurality of predicates. The identifying the one or more applicable files including reading metadata stored in a metadata store that is separate from the database data. The method further includes pruning inapplicable files comprising database data that does not satisfy at least one of the plurality of predicates to create a reduced set of files and reading the reduced set of files to execute the query.

Abstract: Systems, methods, and devices for executing a task on database data in response to a trigger event are disclosed. A method includes executing a transaction on a table comprising database data, wherein executing the transaction comprises generating a new table version. The method includes, in response to the transaction being fully executed, generating a change tracking entry comprising an indication of one or more modifications made to the table by the transaction and storing the change tracking entry in a change tracking stream. The method includes executing a task on the new table version in response to a trigger event.

Abstract: Example systems and methods for cloning catalog objects are described. In one implementation, a method includes creating a copy of a catalog object without copying a data associated with the catalog object by only coping metadata associated with the object. The method further includes modifying, by one or more processors, the data associated with the catalog object independently of the copy of the catalog object such that modified data is not visible to the copy of the catalog object and wherein the modified data is not visible to the catalog object in a database. The method further includes deleting the copy of the catalog object following completion of the modification of the data associated with the catalog object.

Abstract: A method and apparatus managing a set of processors for a set of queries is described. In an exemplary embodiment, a device receives a set of queries for a data warehouse, the set of queries including one or more queries to be processed by the data warehouse. The device further provisions a set of processors from a first plurality of processors, where the set of processors to process the set of queries, and a set of storage resources to store data for the set of queries. In addition, the device monitors a utilization of the set of processors as the set of processors processes the set of queries. The device additionally updates a number of the processors in the set of processors provisioned based on the utilization/Furthermore, the device processes the set of queries using the updated set of processors.

Abstract: Systems, methods, and devices, for managing data skew during a join operation are disclosed. A method includes computing a hash value for a join operation and detecting data skew on a probe side of the join operation at a runtime of the join operation using a lightweight sketch data structure. The method includes identifying a frequent probe-side join key on the probe side of the join operation during a probe phase of the join operation. The method includes identifying a frequent build-side row having a build-side join key corresponding with the frequent probe-side join key. The method includes asynchronously distributing the frequent build-side row to one or more remote servers.

Abstract: Systems, methods, and devices for generating a transactional change tracking summary for a database. A method includes executing a transaction on a table of a database, wherein the table includes a micro-partition and the transaction is executed on the micro-partition. The method includes, in response to the transaction being fully executed, generating a change tracking entry comprising an indication of one or modifications made to the table by the transaction. The method includes storing the change tracking entry in a change tracking stream, wherein the change tracking stream includes one or more sequential change tracking entries that each correspond to a different transaction that is fully executed on the table.

Abstract: A method, apparatus, and system for join operations of a plurality of relations that are distributed over a plurality of storage locations over a network of computing components.

Abstract: A method and apparatus managing a set of processors for a set of queries is described. In an exemplary embodiment, a device receives a set of queries for a data warehouse, the set of queries including one or more queries to be processed by the data warehouse. The device further provisions a set of processors from a first plurality of processors, where the set of processors to process the set of queries, and a set of storage resources to store data for the set of queries. In addition, the device monitors a utilization of the set of processors as the set of processors processes the set of queries. The device additionally updates a number of the processors in the set of processors provisioned based on the utilization/ Furthermore, the device processes the set of queries using the updated set of processors.

Abstract: Example systems and methods for cloning catalog objects are described. In one implementation, a method includes creating a copy of a catalog object without copying a data associated with the catalog object by only coping metadata associated with the object. The method further includes modifying, by one or more processors, the data associated with the catalog object independently of the copy of the catalog object.

Abstract: Replication and failover of database data is disclosed. A method includes replicating database data stored in a primary deployment such that the database data is further stored in a secondary deployment. The method includes executing one or more updates to the database data at the secondary deployment when the primary deployment is unavailable and propagating the one or more updates to the primary deployment when the primary deployment becomes available again. The method includes executing queries on the database data at the primary deployment when the primary deployment is available.

Abstract: Sharing data in a data exchange across multiple cloud computing platforms and/or cloud computing platform regions is described. An example computer-implemented method can include creating a listing in a data exchange, the listing including a data set hosted by a first cloud computing entity. The data set can be shared with a second cloud computing entity.

Abstract: A source table organized into a set of micro-partitions is accessed by a network-based data warehouse. A pruning index is generated based on the source table. The pruning index comprises a set of filters that indicate locations of distinct values in each column of the source table. A query directed at the source table is received at the network-based data warehouse. The query is processed using the pruning index. The processing of the query comprises pruning the set of micro-partitions of the source table to scan for data matching the query, the pruning of the plurality of micro-partitions comprising identifying, using the pruning index, a sub-set of micro-partitions to scan for the data matching the query.

Abstract: Provided herein are systems and methods for an efficient method of replicating share objects to remote deployments. For example, the method may comprise modifying a share object of a first account of a data exchange into a global object wherein the share object includes grant metadata indicating share grants to a set of objects of a database. The method may further comprise creating, in a second account of the data exchange, a local replica of the share object on the remote deployment based on the global object, wherein the second account is located in a remote deployment. The set of objects of the database may be replicated to a local database replica on the remote deployment and the share grants may be replicated to the local replica of the share object.

Abstract: A method of sharing data in a multi-tenant database includes generating a share object in a first account comprising a share role. The method includes associating one or more access rights with the share role, wherein the one or more access rights indicate which objects in the first account are accessible based on the share object. The method includes granting, to a second account, cross-account access rights to the share role or share object in the first account. The method includes receiving a request from the second account to access data or services of the first account. The method further includes providing a response to the second account based on the data or services of the first account.

Abstract: Systems, methods, and devices for generating a materialized view over an external table. A method includes connecting a database platform to an external table such that the database platform has read access for the external table and does not have write access for the external table. The method includes generating, by the database platform, a materialized view over the external table. The method includes receiving a notification that a modification has been made to the external table, the modification comprising one or more of an addition of a file, a deletion of a file, or an update to a file in a source location for the external table. The method includes, in response to the external table being modified, refreshing the materialized view such that the materialized view comprises an accurate representation of the external table.

Abstract: A method for sharing data in a multi-tenant database includes receiving, by a target account of a multiple tenant database, access rights of a share object in a first account of the multiple tenant database, wherein the share object having access rights to a database object of the first account and wherein access to the database object of the first account by the target account is based on the access rights of the share object. The method also includes receiving, by one or more processors of the target account, access rights to an alias object, wherein the alias object references the database object of the first account.

Abstract: Example resource management systems and methods are described. In one implementation, a resource manager is configured to manage data processing tasks associated with multiple data elements. An execution platform is coupled to the resource manager and includes multiple execution nodes configured to store data retrieved from multiple remote storage devices. Each execution node includes a cache and a processor, where the cache and processor are independent of the remote storage devices. A metadata manager is configured to access metadata associated with at least a portion of the multiple data elements.