Abstract: Example resource management systems and methods are described. In one implementation, a resource manager is configured to manage data processing tasks associated with multiple data elements. An execution platform is coupled to the resource manager and includes multiple execution nodes configured to store data retrieved from multiple remote storage devices. Each execution node includes a cache and a processor, where the cache and processor are independent of the remote storage devices. A metadata manager is configured to access metadata associated with at least a portion of the multiple data elements.

Abstract: Techniques for database configurations for remote deployments include a method performed by a data platform executing instructions on at least one processor. The method includes provisioning by at least one hardware processor, a remote deployment of a data platform with a plurality of objects. The plurality of objects includes at least one task object associated with a primary deployment of the data platform. The method further includes detecting using the at least one task object of the plurality of objects, a request to replicate a database stored at the primary deployment of the data platform at the remote deployment. Responsive to the request, database data is pushed from the database stored at the primary deployment to at least a second object of the plurality of objects provisioned at the remote deployment.

Abstract: Aspects of the present disclosure address systems, methods, and devices for tracking object dependencies in a cloud database system. An object dependency created between a referencing object and a referenced object is detected. Based on detecting the object dependency, a dependency record is generated. The dependency record includes dependency information describing the object dependency between the reference object and the referenced object. The dependency record is stored in a database of dependency records.

Abstract: A system and method of encrypting data via public key cryptography with certificate verification of target. The method includes receiving an unsigned certificate signing request (CSR) for a second digital certificate associated with a second application executing on a second client device. The method includes signing, by a processing device of a secret sharing management (SSM) system, the unsigned CSR using a second private key associated with the second client device to generate a signed CSR, the second private key is inaccessible to the second client device. The method includes generating a second digital certificate associated with the second application based on the signed CSR and a different private key associated with the SSM system. The method includes causing the second digital certificate associated with the second application to be stored in a shared data storage available to a first client device.

Abstract: Provided herein are systems and methods for an efficient method of replicating share objects to remote deployments. An example method includes receiving, at a first deployment in a data exchange, a refresh message from a second deployment in the data exchange, wherein the first deployment comprises a database that stores a database object. The method also includes, in response to receiving the refresh message, retrieving, at the first deployment, a share grant from a share object of the first deployment and transmitting, by a processing device of the first deployment, a message including the share grant to the second deployment. The share grant includes a reference to the database object and allows a consumer to use the database object.

Abstract: The subject technology receives a query, the query including a set of statements for performing the query. The subject technology populates a compilation context based at least in part the query. The subject technology invokes a compiler to perform a compilation process based on the compilation context. The subject technology performs a lookup operation on a stored plan cache for an exact match based on information from the compilation context. The subject technology, in response to determining an exact match, determines whether the particular query plan requires re-compilation based on a data dependent optimization. The subject technology determines whether a plan cache entry corresponding to the particular query plan includes a data property constraint. The subject technology determines whether the data property constraint still holds based on a set of data properties.

Abstract: Techniques for executing show commands are described herein. A plurality of navigation steps is utilized, each navigation step corresponding to a different layer in a database structure and each navigation step including an operator to fetch items from a metadata database up to respective bounded limits. Dependency information is also fetched for objects of the specified object type in the show command. After a set of objects from the last layer are processed, memory for the navigation steps is flushed and the next set of objects are processed.

Abstract: Techniques for configuring managed event tables include generating at a first process of an execution node, log data associated with execution of user-defined function (UDF) code. The log data is provided from the first process to a second process of the execution node. The first process is configured as a sub-process of the second process. The log data is formatted using the second process of the execution node, to generate formatted log data. The formatting is based on a configuration of a managed event table that is external to the execution node. The formatted log data is communicated from the second process of the execution node into a managed event table maintained at a computing node that is external to the execution node.

Abstract: A system for sub-table replication within a materialized view of a source data table. A materialized view of a source data table is generated based on a query, the source data table having a plurality of data table rows associated with a plurality of client identifiers and including an entitlements column and data content, the entitlements column including a plurality of provider account identifiers and a plurality of consumer account identifiers. An update to an entitlements table of the source data table is detected where the entitlements table includes a plurality of entitlements table rows and where the entitlements table rows are associated with the client identifiers, the consumer account identifiers, and the provider account identifiers. An update to the source data table based on the first update to the entitlements table is detected, and an incremental replication of the materialized view of the source data table is performed.

Abstract: A system includes at least one hardware processor of a computing node and at least one memory storing instructions that cause the at least one hardware processor to perform operations. The operations include instantiating a user code runtime to execute within a sandbox process. The sandbox process configures access by the user code runtime to an operating system (OS) kernel of the computing node. The OS kernel is configured with one or more filtering policies. A determination is performed of whether a system call received by the OS kernel violates the one or more filtering policies. The system call is triggered by at least one operation of the user code runtime. A tracing event is instantiated to trace execution of the system call based on the determination.

Abstract: Techniques for automated query retry in a database platform include decoding, by at least one hardware processor, a query directed to database data from a client account of a database platform. The method further includes decoding, by the at least one hardware processor, an indication that execution of the query on at least one computing node of the database platform results in a failed execution. The method further includes configuring a processing loop with continuous retry executions of the query on the at least one computing node based on the indication. The method further includes exiting the processing loop based on detecting a retry execution of the continuous retry executions results in at least one successful execution of the query. The method includes logging each attempt to execute the query during the continuous retry executions in a query status log until the at least one successful execution of the query.

Abstract: The subject technology receives first metadata corresponding to a set of micro-partitions. The subject technology stores a first data structure and a second data structure in storage as a first file and a second file, first data structure including the first metadata and a second data structure including second metadata, the first metadata corresponding to a set of micro-partitions, the second metadata for a grouping of the first metadata, the second data structure including information associating the second metadata to the first metadata. The subject technology stores third metadata for a table, the third metadata comprising information about data stored in a micro-partition of the table.

Abstract: As described herein, a N-Gram index may be created and the search may be conducted using the index, which will lead to faster search results. The N-Gram index may also include partial N-Gram components to capture more relevant data. Moreover, as described herein, the search may also take into account recent log data that has not yet been indexed. Techniques for building an index store using log data and efficiently searching the index store and log data to process search requests are described herein.

Abstract: Systems and methods for managing membership in a private data exchange are provided herein. In one embodiment, the method includes generating a plurality of listings in a data exchange. A first listing of the plurality of listings may include a reference to a first version of shared data within a first database, where the first version is one of a plurality of versions of the shared data within the first database. The method further includes receiving a request from a member of the data exchange for access to the first listing and limiting, by a processing device, access for the member to the first version of the shared data of the first database referenced by the first listing.

Abstract: Techniques for configuring query result information associated with query result data include configuring execution of a query using a plurality of execution stages. Query result data is generated based on the execution of the query. The query result data includes a plurality of data portions. Each data portion of the plurality of data portions is obtained at a corresponding execution stage of the plurality of execution stages. Query result information is generated based on the execution of the query. The query result information includes authentication information associated with accessing the plurality of data portions.

Abstract: Signing source code using a vault device is described. An example method can include receiving, with a client device, source code that is to be committed to a repository. The method further can include sending, with a processing device to a vault device, a request to sign the source code. The method can further include receiving, from the vault device, a signed commit of the source code. In addition, the method can include pushing the source code and the signed commit to the repository, wherein the repository commits the source code and signed commit.

Abstract: Replication and failover of database data is disclosed. A method includes copying a first dataset of a first account stored in a primary deployment to a secondary deployment. The method includes determining that the first dataset stored in the primary deployment is unavailable. The method includes executing a first transaction on the first dataset at the secondary deployment in response to determining that the first dataset stored in the primary deployment is unavailable. The method includes executing a second transaction on a second dataset of a second account at the primary deployment while the first dataset of the first account stored in the primary deployment is unavailable.

Abstract: Generating semantic names for a data set is described. An example method can include retrieving data from a data set, the data organized in a plurality of columns. The method may also include generating one or more candidate semantic categories for that column, wherein each of the one or more candidate semantic categories has a corresponding probability for each of the columns. The method may also further include creating a feature vector for each column from the one or more column candidate semantic categories and the corresponding probabilities. Additionally, the method may also include selecting, for each column, a column semantic category from the one or more candidate semantic categories using at least the feature vector and a trained machine learning model.

Abstract: A method for a multi-cluster warehouse includes allocating a plurality of compute clusters as part of a virtual warehouse. The compute clusters are used to access and perform queries against one or more databases in one or more cloud storage resources. The method includes providing queries for the virtual warehouse to each of the plurality of compute clusters. Each of the plurality of compute clusters of the virtual warehouse receives a plurality of queries so that the computing load is spread across the different clusters. The method also includes dynamically adding compute clusters to and removing compute clusters from the virtual warehouse as needed based on a workload of the plurality of compute clusters.

Abstract: A system and method for automatically routing datasets based on input and output requirements of information is disclosed. The example method includes obtaining a list of modules executing on one or more processing devices, each module to generate an output dataset of a respective output requirement based on an input dataset of a respective input requirement. The method also includes establishing connections between the modules based on the input requirements and output requirements, including a first connection between a first module and a third module and a second connection between a second module and the third module. The method also includes receiving a first output dataset from the first module and receiving a second output dataset from the second module and, responsive to determining that the first module reported an error, routing the second output dataset to the third module without routing the first output dataset to the third module.