Abstract: In an embodiment, a data platform creates an application in a data-provider account. The application includes one or more APIs corresponding to one or more underlying code blocks. The data platform shares provider data with the application in the data-provider account, and also installs, in a data-consumer account, an application instance of the application. The application instance includes one or more APIs corresponding to the one or more APIs in the application in the data-provider account. The data platform shares consumer data with the application instance in the data-consumer account, and invokes one or more of the APIs of the application instance to execute respective associated underlying code blocks, which are not visible to the data-consumer account. The data platform also saves output of the one or more respective associated underlying code blocks locally within the data-consumer account.

Abstract: A source table organized into a set of micro-partitions is accessed by a network-based data warehouse. A pruning index is generated based on the source table. The pruning index comprises a set of filters that indicate locations of distinct values in each column of the source table. A query directed at the source table is received at the network-based data warehouse. The query is processed using the pruning index. The processing of the query comprises pruning the set of micro-partitions of the source table to scan for data matching the query, the pruning of the plurality of micro-partitions comprising identifying, using the pruning index, a sub-set of micro-partitions to scan for the data matching the query.

Abstract: A query directed at a source table organized into a set of batch units is received. The query comprises a regular expression search pattern. The regular expression search pattern is converted to a pruning index predicate comprising a set of substring literals extracted from the regular expression search pattern. A set of N-grams is generated based on the set of substring literals extracted from the regular expression search pattern. A pruning index associated with the source table is accessed. The pruning index indexes distinct N-grams in each column of the source table. A subset of batch units to scan for data matching the query are identified based on the pruning index and the set of N-grams. The query is processed by scanning the subset of batch units.

Abstract: A system and method for pruning data based on metadata. The method may include receiving a query that includes a plurality of predicates and identifying one or more applicable files including database data satisfying at least one of the plurality of predicates. The identifying the one or more applicable files including reading metadata stored in a metadata store that is separate from the database data. The method further includes pruning inapplicable files including database data that does not satisfy at least one of the plurality of predicates to create a reduced set of files and reading the reduced set of files to execute the query.

Abstract: A method of sharing data in a multi-tenant database includes inspecting, by a processing device of a multiple tenant database, a sharer account to determine an existence of a link between an alias object in a target account and a database, wherein the database is linked to a first role object included in a share object in the sharer account. The method includes granting a second role object, in the target account, access rights to the alias object, wherein the first role object having one or more grants to one or more resources of the sharer account, and wherein the target account accesses the one or more resources using the one or more grants of the share object and using the alias object without at least one of copying the one or more resources or transmitting the one or more resources.

Abstract: The subject technology receives a query, the query including a set of statements for performing the query. The subject technology populates a compilation context based at least in part the query. The subject technology invokes a compiler to perform a compilation process based on the compilation context. The subject technology performs a lookup operation on a stored plan cache for an exact match based on information from the compilation context. The subject technology, in response to determining an exact match, determines whether the particular query plan requires re-compilation based on a data dependent optimization. The subject technology determines whether a plan cache entry corresponding to the particular query plan includes a data property constraint. The subject technology determines whether the data property constraint still holds based on a set of data properties.

Abstract: Example resource management systems and methods are described. In one implementation, a resource manager is configured to manage data processing tasks associated with multiple data elements. An execution platform is coupled to the resource manager and includes multiple execution nodes configured to store data retrieved from multiple remote storage devices. Each execution node includes a cache and a processor, where the cache and processor are independent of the remote storage devices. A metadata manager is configured to access metadata associated with at least a portion of the multiple data elements.

Abstract: Hybrid tables can be used in different use-case scenarios. Hybrid tables provide a flexible mechanism to support files and data in different formats while providing access to the different types of data as part of one table. This flexibility can allow the use of hybrid tables in data lake or other similar environments.

Abstract: Example caching systems and methods are described. In one implementation, a method identifies multiple files used to process a query and distributes each of the multiple files to a particular execution node to execute the query. Each execution node determines whether the distributed file is stored in the execution node's cache. If the execution node determines that the file is stored in the cache, it processes the query using the cached file. If the file is not stored in the cache, the execution node retrieves the file from a remote storage device, stores the file in the execution node's cache, and processes the query using the file.

Abstract: A method for sharing data in a multi-tenant database includes generating a share object in a first account comprising a share role. The method includes associating one or more access rights with the share role, wherein the one or more access rights indicate which objects in the first account are accessible based on the share object. The method includes granting, to a second account, cross-account access rights to the share role or share object in the first account. The method includes receiving a request from the second account to access data or services of the first account. The method further includes providing a response to the second account based on the data or services of the first account.

Abstract: A data platform creates an application in a data-provider account, where the application includes one or more application programming interfaces (APIs) corresponding to one or more underlying code blocks. The data platform shares homomorphically encrypted provider data with the application in the data-provider account. The data platform installs, in a data-consumer account, an application instance of the application. The data platform shares homomorphically encrypted consumer data with the application instance in the data-consumer account. The data platform invokes one or more of the APIs of the application instance to execute respective associated underlying code blocks, which are not visible to the data-consumer account, and which operate on the shared homomorphically encrypted provider data and the shared homomorphically encrypted consumer data. The data platform saves homomorphically encrypted output of the one or more respective associated underlying code blocks locally within the data-consumer account.

Abstract: A system and method of scaling capacity of data warehouses to user-defined levels. The method includes monitoring a processing of one or more queries by a data warehouse, the data warehouse including one or more processing resources and a storage platform. The method includes receiving a request to process database data stored on the storage platform. The method includes determining a performance of the processed request as observed by a user device. The method includes adjusting, by a processing device, a timing associated with the processing of the one or more queries based on the performance of the processed request.

Abstract: A data platform for managing an application as a first-class database object. The data platform includes at least one processor and a memory storing instructions that cause the at least one processor to perform operations including detecting a data request from a browser for a data object located on the data platform, executing a stored procedure, the stored procedure containing instructions that cause the at least one processor to perform additional operations including instantiating a User Defined Function (UDF) server, an application engine, and the application within a security context of the data platform based on a security policy determined by an owner of the data object. The data platform then communicates with the browser using the application engine as a proxy server.

Abstract: Example caching systems and methods are described. In one implementation, a method identifies multiple files used to process a query and distributes each of the multiple files to a particular execution node to execute the query. Each execution node determines whether the distributed file is stored in the execution node's cache. If the execution node determines that the file is stored in the cache, it processes the query using the cached file. If the file is not stored in the cache, the execution node retrieves the file from a remote storage device, stores the file in the execution node's cache, and processes the query using the file.

Abstract: Example caching systems and methods are described. In one implementation, a method identifies multiple files used to process a query and distributes each of the multiple files to a particular execution node to execute the query. Each execution node determines whether the distributed file is stored in the execution node's cache. If the execution node determines that the file is stored in the cache, it processes the query using the cached file. If the file is not stored in the cache, the execution node retrieves the file from a remote storage device, stores the file in the execution node's cache, and processes the query using the file.

Abstract: Example caching systems and methods are described. In one implementation, a method identifies multiple files used to process a query and distributes each of the multiple files to a particular execution node to execute the query. Each execution node determines whether the distributed file is stored in the execution node's cache. If the execution node determines that the file is stored in the cache, it processes the query using the cached file. If the file is not stored in the cache, the execution node retrieves the file from a remote storage device, stores the file in the execution node's cache, and processes the query using the file.

Abstract: A source table organized into a set of micro-partitions is accessed by a network-based data warehouse. A pruning index is generated based on the source table. The pruning index comprises a set of filters that indicate locations of distinct values in each column of the source table. A query directed at the source table is received at the network-based data warehouse. The query is processed using the pruning index. The processing of the query comprises pruning the set of micro-partitions of the source table to scan for data matching the query, the pruning of the plurality of micro-partitions comprising identifying, using the pruning index, a sub-set of micro-partitions to scan for the data matching the query.

Abstract: Caching systems and methods are described. In one implementation, a method identifies multiple files used to process a query and distributes each of the multiple files to a particular execution node to execute the query. Each execution node determines whether the distributed file is stored in the execution node's cache. If the execution node determines that the file is stored in the cache, it processes the query using the cached file. If the file is not stored in the cache, the execution node retrieves the file from a remote storage device, stores the file in the execution node's cache, and processes the query using the file.

Abstract: A method implementing a fault-tolerant data warehouse using availability zones includes allocating a plurality of processing units to a data warehouse, the processing units located in different availability zones, an availability zone comprising one or more data centers. The method further includes routing a query to a processing unit within the data warehouse, the query having a common session identifier with a query previously provided to the processing unit, the processing unit determined to be caching a data segment associated with a cloud storage resource independent of the plurality of processing units. The method further includes, as a result of monitoring a number of queries running at an input degree of parallelism, determining that the processing capacity of the processing units has reached a threshold; and changing a total number of processing units using the input degree of parallelism and the number of queries.

Abstract: A method of sharing data in a multi-tenant database includes inspecting, by a processing device of a multiple tenant database, a sharer account to determine a presence of a grant to a second role object, in a target account, of access rights to a first role object included in the sharer account. The method includes granting the second role object, in the target account, access rights to an alias object. The first role object having one or more grants to the one or more resources of the sharer account. The target account accesses the one or more resources using the one or more grants of the first role object and using the alias object without at least one of copying the one or more resources or transmitting the one or more resources.