Abstract: Interleaved infrastructure can interleave fragment execution during query compilation. For example, compilation can be paused, and a plan fragment is executed. The results of the plan fragment can be fed back into the compiler, which can then restart the compilation and modify the compilation based on the results of the plan fragment execution. The interleaved workflow provides a flexible, robust infrastructure, which can enable and improve a variety of database features and query optimizations.

Abstract: A method implementing a fault-tolerant data warehouse including allocating a plurality of processing units to a data warehouse, the processing units located in different availability zones, an availability zone comprising one or more data centers. The method further includes, as a result of monitoring a number of queries running at an input degree of parallelism on the plurality of processing units of the data warehouse, determining that the number of queries is serviceable by one fewer processing units. The method further includes routing a query from a first processing unit to a second processing unit within the data warehouse, the query having a common session identifier with another query previously provided to the second processing unit, the second processing unit determined to be caching a data segment associated with a cloud storage resource, usable by the query, and removing the first processing unit from the data warehouse.

Abstract: Systems, methods, and devices for tracking a series of changes to database data are disclosed. A method includes executing a transaction to modify data in a micro-partition of a table of a database by generating a new micro-partition that embodies the transaction. The method includes associating transaction data with the new micro-partition, wherein the transaction data comprises a timestamp when the transaction was fully executed, and further includes associating modification data with the new micro-partition that comprises an indication of one or more rows of the table that were modified by the transaction. The method includes joining the transaction data with the modification data to generate joined data and querying the joined data to determine a listing of intermediate modifications made to the table between a first timestamp and a second timestamp.

Abstract: Techniques for configuring a managed event table (MET) include detecting, by at least one hardware processor, a query for the MET. The query received at a first computing node of a network-based database system. The method includes retrieving via an ingestion function configured at the first computing node, reformatted data from a dedicated storage location of a first process into the MET. The reformatted data is based on log data associated with a second process. The first process and the second process are executing at a second computing node of the network-based database system. The method includes processing the query using the reformatted data in the MET.

Abstract: A method for a first set of processors and a second set of processors comprises, the first set of processors processing a set of queries, as a result of a change in utilization of the first set of processors, processing the set of queries using the second set of processors. The change in processors is independent of a change in storage resources, the storage resources shared by the first set of processors and the second set of processors.

Abstract: A method and apparatus for managing a set of storage resources for a set of queries is described. In an exemplary embodiment, a method provisions processing resources of an execution platform and provisions storage resources of a storage platform. The execution platform uses the storage platform, which is shared with the execution platform, to process the set of queries. The method changes a number of the storage resources provisioned for the storage platform based on a storage capacity utilization by the set of queries of the storage resources. The method changes the number of the storage resources independently to a change of the processing resources in the execution platform. The method processes the set of queries using the changed number of the storage resources provisioned for the storage platform.

Abstract: A database system can configure network devices, such as a primary database in a multi-tenant deployment and a secondary database in a private deployment, to send and receive sequence messages, such as input data indicative of a selection of a link. The database system can create a secure share area in the private deployment in response to receiving the input data indicative of the selection of the link. The database system can replicate the data from the multi-tenant deployment to the secure share area in the private deployment and share the replicated data from the secure share area to the secondary database hosted in the private deployment.

Abstract: System and method for efficiently querying external tables are described herein. In an embodiment, a database platform receives a query that is directed at least in part to external data in an external table stored on a data storage platform that is external to the database platform. The external table includes a plurality of partitions. The database platform identifies, from external-table metadata, a subset of the plurality of partitions of the external table as including data that potentially satisfies the query. The external-table metadata is stored by the database platform. The database platform identifies data that satisfies the query by scanning the identified subset of the partitions, and responds to the query at least in part with the identified data that satisfies the query.

Abstract: A method of preventing queries from accessing database files based on metadata. The method includes determining a first metadata associated with a particular file and a second metadata associated with a changed version of the particular file. The method includes directing, based on the first metadata associated with the particular file, a first query for the particular file to a first file that is associated with the particular file. The method includes preventing a second query for the particular file from accessing the particular file by directing, based on the second metadata associated with the changed version of the particular file, the second query to the changed version of the particular file instead of the particular file.

Abstract: Systems of methods of directing requests to databases based on client account association is disclosed. A method includes determining a first client account associated with a first request to perform a first transaction. The method includes determining a second client account associated with a second request to perform a second transaction. The method includes selecting, for the first request, a primary deployment that stores a first dataset responsive to determining the first client account associated with the first request. The method includes selecting, for the second request, a secondary deployment that stores a second dataset that includes the first dataset responsive to determining the second client account associated with the second request. The method includes executing, while the primary deployment and the secondary deployment are both available, the first transaction on the first dataset at the primary deployment and the second transaction on the second dataset at the secondary deployment.

Abstract: A method includes ingesting, by at least one hardware processor, database data within a staging table of a database. The method further includes generating a task based on at least one data processing statement received from a user account. The method further includes executing the task on the database data in the staging table to generate modified data. The method further includes, in response to the task being fully executed on the database data, storing the modified data in a target table. In some aspects, the target table is configured within the database. The method further includes generating a stream of one or more transactions committed during the execution of the task to generate the modified data.

Abstract: Techniques for active invalidation of cache entries are described. A first timestamp and a second timestamp are stored by a compute instance, the first timestamp corresponding to an initial time the computing instance read from a version update log in a database and the second timestamp corresponding to a last time the computing instance read from the version update log in the database. The compute instance can read records from the version update log that are associated with version timestamps greater than the second timestamp. The compute instance can invalidate a first set of cache entries with older versions stored in a cache based on reading the records, retaining a second set of cache entries, and then execute a query using the second set of cache entries.

Abstract: Techniques for creating, sharing, and using bundles (also referred to as packages) in a multi-tenant database are described herein. A bundle is a schema object with associated hidden schemas. A bundle can be created by a provider user and can be shared with a plurality of consumer users. The bundle can be used to enable code sharing and distribution without losing control while maintaining security protocols.

Abstract: Interleaved infrastructure can interleave fragment execution during query compilation. For example, compilation can be paused, and a plan fragment is executed. The results of the plan fragment can be fed back into the compiler, which can then restart the compilation and modify the compilation based on the results of the plan fragment execution. The interleaved workflow provides a flexible, robust infrastructure, which can enable and improve a variety of database features and query optimizations.

Abstract: Sharing data in a data exchange across multiple cloud computing platforms is described. An example method includes copying, to a first cloud computing entity using a consumer account of the first cloud computing entity, a first subset of a data set associated with a provider account of a second cloud computing entity, wherein the provider account of the second cloud computing entity does not have access to the first cloud computing entity, and verifying existence of the first subset of the data set at the first cloud computing entity.

Abstract: Techniques for creating, sharing, and using bundles (also referred to as packages) in a multi-tenant database are described herein. The bundle includes an interface directly accessible to the consumer account and a hidden bundle schema not directly accessible by the consumer account. A consumer account can upgrade from a first version to a second version by way of an intermediate version.

Abstract: The subject technology serializes, by at least one hardware processor, non-primary key data of column-organized data into compressed serialized value data that is in a row-organized sequence, the compressed serialized value data compressed using at least one bitmap, the non-primary key data comprising a schema identifier, the column-organized data being stored in a columnar database system, the column-organized data comprising primary key data and the non-primary key data. The subject technology stores the compressed serialized value data in a key-value data store of a key-value database system, the key-value database system processing key-value data in a key-value format. The subject technology receives a query by the columnar database system. The subject technology deserializes a portion of the compressed serialized value data that corresponds to the query. The subject technology processes the query using the columnar database system.

Abstract: A method, apparatus, and system for join operations of a plurality of relations that are distributed over a plurality of storage locations over a network of computing components.

Abstract: Techniques for configuring event tables include retrieving, by at least one hardware processor of a computing node, log data at a first process of the computing node. The log data is associated with a function executing at a second process of the computing node. The log data is revised using a table stage to generate revised log data. The table stage is configured as a dedicated storage location of the first process. The revising includes a data enrichment process based on metadata associated with execution of the function at the second process. The revised log data is ingested into an event table.

Abstract: Provided herein are systems and methods for providing a secure and efficient way to manage a data exchange's availability and manage the visibility of data listings of the data exchange. For example, the method may comprise specifying, by an exchange admin of the data exchange, a set of regions where the data exchange is available, each of the set of regions comprising one or more remote deployments. The method may further comprise specifying by a data provider, one or more of the set of regions where a data listing owned by the data provider is visible. Upon receiving a request to access the data listing from a remote deployment of the one or more regions, the data provider may determine whether to reject or fulfill the request. In response to determining that the request should be fulfilled, replicating data of the data listing to the remote deployment.