Abstract: Provided herein are systems and methods for configuring managed event tables. A system includes at least one hardware processor coupled to a memory and configured to collect using an event table instance within a sandbox process, log data generated by a UDF during execution of the UDF code. The log data is provided from the sandbox process to an execution node process of the execution node. The log data is formatted using the execution node process, to generate formatted log data. The formatting is based on a configuration of a managed event table that is external to the execution node. The at least one hardware processor further causes ingestion of the formatted log data from the execution node process into the managed event table.

Abstract: Embodiments of the present disclosure provide techniques for defining database roles to allow sharing of the objects within a database in a more granular fashion. A set of database roles is created within a database container of a provider account, each database role comprising a separate object from any of a plurality of objects within the database container. A set of grants to a particular subset of the plurality of data objects of the database container is assigned to each of the set of database roles. A share object to which each of the set of database roles has been granted is mounted within a consumer account, thereby generating an imported database container having an imported copy of each of the set of database roles within the consumer account. Certain imported copies of the set of database roles are granted to account level roles of the consumer account.

Abstract: A system and method for managing data storage and data access with querying data in a distributed system without buffering the results on intermediate operations in disk storage.

Abstract: A method and apparatus managing a set of processors for a set of queries is described. In an exemplary embodiment, a device receives a set of queries for a data warehouse, the set of queries including one or more queries to be processed by the data warehouse. The device further provisions a set of processors from a first plurality of processors, where the set of processors to process the set of queries, and a set of storage resources to store data for the set of queries. In addition, the device monitors a utilization of the set of processors as the set of processors processes the set of queries. The device additionally updates a number of the processors in the set of processors provisioned based on the utilization/Furthermore, the device processes the set of queries using the updated set of processors.

Abstract: Techniques for automated query retry in a database platform include decoding, by at least one hardware processor, a query directed to database data from a client account of a database platform. The method further includes decoding, by the at least one hardware processor, an indication that execution of the query on at least one computing node of the database platform results in a failed execution. The method further includes configuring a processing loop with continuous retry executions of the query on the at least one computing node based on the indication. The method further includes exiting the processing loop based on detecting a retry execution of the continuous retry executions results in at least one successful execution of the query. The method includes logging each attempt to execute the query during the continuous retry executions in a query status log until the at least one successful execution of the query.

Abstract: A system includes one or more processors and data storage containing instructions executable by the one or more processors to perform operations. The operations include detecting a first executed transaction causing a first modification to table data stored in a table. The table data is associated with a corresponding metadata file with metadata information of the table. A new metadata file is generated responsive to the first executed transaction. The new metadata file includes the metadata information and additional metadata associated with the first modification. A second executed transaction causing a second modification to the table data is detected. The table data is updated with a change tracking column. The change tracking column includes lineage of executed transactions on the table data. The lineage indicates at least the first transaction and the second transaction.

Abstract: Hybrid tables can be used in different use-case scenarios. Hybrid tables provide a flexible mechanism to support files and data in different formats while providing access to the different types of data as part of one table. This flexibility can allow the use of hybrid tables in data lake or other similar environments.

Abstract: Providing access to metrics on a private data exchange is described. An example computer-implemented method can include executing one or more queries against data referenced by a data listing of a data exchange, the data exchange comprising a plurality of data listings provided by one or more data providers, gathering consumer information based on the one or more queries, assigning privileges that grant access to the consumer information to a share object, in response to the share object being added to a list of consumed share objects of a data provider that provided the data listing, generating a database that references the consumer information based on the share object, and granting access to the database to the data provider that provided the data listing.

Abstract: Provided herein are systems and methods for an efficient method of replicating share objects to remote deployments. For example, the method may include generating a global representation of a share object of a first database account located in a first region. The share object includes grant metadata associated with a set of objects of a database located in the first region and associated with the first database account. The method may further include, in response to a database refresh command received from a second database account associated with a database replica located in a second region, replicating the set of objects of the database to the database replica. The method may further include, in response to a share refresh command received from the second database account, replicating the grant metadata to a share object replica located in the second region.

Abstract: Systems, methods, and devices for incrementally refreshing a materialized view are disclosed. A method includes generating a materialized view based on a source table. The method includes merging the source table and the materialized view to generate a merged table to identify whether an update has been executed on the source table that is not reflected in the materialized view. The method includes, in response to detecting an update made to the source table that is not reflected in the materialized view, applying the update to the materialized view.

Abstract: Hybrid tables can be used in different use-case scenarios. Hybrid tables provide a flexible mechanism to support files and data in different formats while providing access to the different types of data as part of one table. This flexibility can allow the use of hybrid tables in data lake or other similar environments.

Abstract: Sharing data in a data exchange across multiple cloud computing platforms and/or cloud computing platform regions is described. An example computer-implemented method can include receiving data sharing information from a data provider for sharing a data set in a data exchange from a first cloud computing entity to a set of second cloud computing entities. In response to receiving the data sharing information, the method may also include creating an account with each of the set of second cloud computing entities. The method may also further include sharing the data set from the first cloud computing entity with the set of second cloud computing entities using at least the corresponding account of that second cloud computing entity.

Abstract: A method of operating a data exchange includes creating a first listing referencing data of a first database of a plurality of databases, wherein the first listing comprises access controls and a data share associated with a first user, the access controls defining portions of the first database that are accessible by a second user, receiving a request from the second user for a bidirectional share between the portions of the first database that are accessible by the second user and portions of a second database controlled by the second user, and receiving an instruction from the first user to perform a database operation referencing data of the bidirectional share between the portions of the first database that are accessible by the second user and the portions of the second database controlled by the second user.

Abstract: Techniques for database configurations for remote deployments include a method performed by a data platform executing instructions on at least one processor. The method includes provisioning by at least one hardware processor, a remote deployment of a data platform with a plurality of objects. The plurality of objects includes at least one task object associated with a primary deployment of the data platform. The method further includes detecting using the at least one task object of the plurality of objects, a request to replicate a database stored at the primary deployment of the data platform at the remote deployment. Responsive to the request, database data is pushed from the database stored at the primary deployment to at least a second object of the plurality of objects provisioned at the remote deployment.

Abstract: Disclosed herein are systems and methods for database replication to a remote deployment with automated fulfillment. In an embodiment, a data platform provisions a remote-deployment account of a data provider at a remote deployment of the data platform with a set of one or more replication-preparation objects associated with a primary-deployment database that is resident in a primary-deployment account of the data provider at a primary deployment of the data platform. After the provisioning, the data platform detects receipt of a database-replication request associated with a data consumer, requesting availability of a local instance of the primary-deployment database in the remote-deployment account of the data provider. The data platform responsively performs one or more remote-deployment-activation operations, which makes available to the data consumer a local instance of the primary-deployment database in the remote-deployment account of the data provider at the remote deployment of the data platform.

Abstract: Example systems and methods for cloning catalog objects are described. In one implementation, a method identifies an original catalog object associated with data and creates a duplicate copy of the original catalog object without copying the data itself. The method allows access to the data using the duplicate catalog object and supports modifying the data associated with the original catalog object independently of the duplicate catalog object. The duplicate catalog object can be deleted upon completion of modifying the data associated with the original catalog object.

Abstract: Example systems and methods for cloning catalog objects are described. In one implementation, a method includes copying first metadata of a first catalog object to create second metadata of a second catalog object such that a second portion of the second metadata and a first portion of the first metadata are associated with same data of a same data file, and generating a response to a data storage or data retrieval request directed to the first catalog object by accessing the data file using the second metadata of the second catalog object.

Abstract: A method, apparatus, and system including performing a partitioning function on a first table of a join query, determining a size of the first table based on the partitioning function, and distributing the first table to a set of nodes of a computing cluster based on the size of the first table.

Abstract: Embodiments of the present disclosure enable users of a data sharing system to build native applications that can be shared with other users of the data sharing system. The native applications can be published and discovered in the data sharing system like any other data listing, and consumers can install them in their local data sharing system account to serve their data processing needs. A provider may define an installation script for installing an application and create a share object to which the installation script may be attached. In response to an imported database being created in a consumer account based on the share object, a native application framework may automatically execute the installation script in the consumer account and may create a set of database roles to manage execution of the application in the consumer account.

Abstract: Disclosed herein are systems and methods for database replication to a remote deployment with automated fulfillment. In an embodiment, a data platform provisions a remote-deployment account of a data provider at a remote deployment of the data platform with a set of one or more replication-preparation objects associated with a primary-deployment database that is resident in a primary-deployment account of the data provider at a primary deployment of the data platform. After the provisioning, the data platform detects receipt of a database-replication request associated with a data consumer, requesting availability of a local instance of the primary-deployment database in the remote-deployment account of the data provider. The data platform responsively performs one or more remote-deployment-activation operations, which makes available to the data consumer a local instance of the primary-deployment database in the remote-deployment account of the data provider at the remote deployment of the data platform.