Abstract: Example resource management systems and methods are described. In one implementation, a resource manager is to be coupled to one or more virtual data warehouse instances that each comprise a plurality of computing resources. A resource manager receives a query directed to database data. The resource manager determines a plurality of tasks associated with the query. The resource manager distributes the plurality of tasks among the one or more virtual data warehouse instances to cause each computing resource of the plurality of computing resources of the one or more virtual data warehouse instances to process a respective task of the plurality of tasks. The resource manager receives results obtained from the processing of the plurality of tasks. The resource manager uses the results to respond to the query.

Abstract: Techniques for continuous ingestion of files using custom file formats are described. A custom file format may include formats not natively supported by a data system. Unstructured files (e.g., images) may also be considered custom file formats. A custom file format may be set using a user defined table function and scanner options.

Abstract: A database platform receives an object identifier from a client in association with a database session. The client is associated with a customer account of the database platform, and the database session is associated with the client. The customer account includes multiple disjoint account-level namespaces, each of which represents a distinct context for resolution of object identifiers, such that matching object identifiers in different account-level namespaces in the customer account do not collide with respect to object-identifier resolution. The database platform determines that the object identifier does not specify an account-level namespace, and responsively resolves the object identifier with reference to a current account-level namespace of the database session by identifying an object corresponding to the object identifier in the customer account.

Abstract: Automated query retry includes selecting a first node of a plurality of nodes of an execution platform in response to a query. The first node is configured with a first version of a database platform. A first execution of the query is performed using the first version of the database platform at the first node. The method further includes detecting the first execution results in a failed execution. A second execution of the query is scheduled. The second execution uses at least one of the first version of the database platform or a second version of the database platform configured at a second node of the plurality of nodes. The second version is released after the first version. A cause of the failed execution of the query at the first node is determined based at least in part on a result of the second execution of the query.

Abstract: Example caching systems and methods are described. In one implementation, a method identifies multiple files used to process a query and distributes each of the multiple files to a particular execution node to execute the query. Each execution node determines whether the distributed file is stored in the execution node's cache. If the execution node determines that the file is stored in the cache, it processes the query using the cached file. If the file is not stored in the cache, the execution node retrieves the file from a remote storage device, stores the file in the execution node's cache, and processes the query using the file.

Abstract: Sharing data in a data exchange across multiple cloud computing platforms and/or cloud computing platform regions is described. An example computer-implemented method can include creating a listing in a data exchange, the listing including a data set hosted by a first cloud computing entity. The data set can be shared with a second cloud computing entity. The method further includes receiving a request associated with a customer account of the second cloud computing entity to access the data set of the listing hosted by the first cloud computing entity and replicating at least a subset of the data set of the listing from the first cloud computing entity to a provider account at the second cloud computing entity to be accessible by the customer account at the second cloud computing entity.

Abstract: Disclosed herein are systems and methods for efficiently querying external tables. In an embodiment, a database platform receives a query that is directed at least in part to external data in an external table stored on a data storage platform that is external to the database platform. The external table includes a plurality of partitions. The database platform identifies, from external-table metadata, a subset of the plurality of partitions of the external table as including data that potentially satisfies the query. The external-table metadata is stored by the database platform. The database platform identifies data that satisfies the query by scanning the identified subset of the partitions, and responds to the query at least in part with the identified data that satisfies the query.

Abstract: A method, apparatus, and system for join operations of a plurality of relations that are distributed over a plurality of storage locations over a network of computing components.

Abstract: A method includes detecting, by at least one hardware processor, a change request for a table of a database, the table comprising a plurality of micro-partitions. A transaction associated with the change request is executed at a first timestamp. The transaction causes replacement of a first micro-partition of the plurality of micro-partitions with a second micro-partition. A change tracking column is generated in the second micro-partition. The change tracking column comprises metadata for the transaction. A delta for the table between the first timestamp and a second timestamp is generated using the metadata in the change tracking column. The delta indicates changes made to one or more rows of the table between the first time stamp and the second timestamp.

Abstract: Systems, methods, and devices for retrying a query. A method includes receiving a query directed to database data and assigning execution of the query to one or more execution nodes of an execution platform, the one or more execution nodes configured to execute the query on a first version of a database platform. The method includes determining that execution of the query was unsuccessful. The method includes assigning a first retry execution of the query to the one or more execution nodes of the execution platform and determining whether a regression or an intermittent fault caused the execution of the query to be unsuccessful based at least in part on whether the first retry execution of the query was successful or unsuccessful.

Abstract: Example systems and methods for cloning catalog objects are described. In one implementation, a method includes creating a second catalog object by mapping a second portion of second metadata of the second catalog object to same data of a same data file as a first portion of first metadata of a first catalog object, and, in response to a data storage or data retrieval request directed to the second catalog object, deleting data associated with the second portion of the second metadata from the second catalog object independently of the first catalog object.

Abstract: A system includes one or more processors and data storage containing instructions executable by the one or more processors to perform operations. The operations include detecting a first executed transaction causing a first modification to table data stored in a table. The table data is associated with a corresponding metadata file with metadata information of the table. A new metadata file is generated responsive to the first executed transaction. The new metadata file includes the metadata information and additional metadata associated with the first modification. A second executed transaction causing a second modification to the table data is detected. The table data is updated with a change tracking column. The change tracking column includes lineage of executed transactions on the table data. The lineage indicates at least the first transaction and the second transaction.

Abstract: Systems, methods, and devices for sharing materialized views in multiple tenant database systems. A method includes defining a materialized view over a source table that is associated with a first account of a multiple tenant database. The method includes defining cross-account access rights to the materialized view to a second account such that that second account can read the materialized view without copying the materialized view. The method includes modifying the source table for the materialized view. The method includes identifying whether the materialized view is stale with respect to the source table by merging the materialized view and the source table.

Abstract: A database system can configure network devices, such as a primary database in a multi-tenant deployment and a secondary database in a private deployment, to send and receive sequence messages, such as input data indicative of a selection of a link. The database system can create a secure share area in the private deployment in response to receiving the input data indicative of the selection of the link. The database system can replicate the data from the multi-tenant deployment to the secure share area in the private deployment and share the replicated data from the secure share area to the secondary database hosted in the private deployment.

Abstract: Systems and methods for managing membership in a private data exchange are provided herein. In one embodiment, the method includes receiving a request for access to a first listing of a data exchange, each listing of the data exchange comprising version metadata. The method further includes, in response to the request, accessing a first version of a data set referenced by the first listing, wherein the first version of the data set comprises a first structure defined by first version metadata, a second listing of the data exchange references a second version of the data set, the second version of the data set comprising a second structure defined by second version metadata, and the second structure is incompatible with the first structure.

Abstract: The subject technology receives a command to commit a table in a different table format on an external volume. The subject technology generates a first snapshot of the table on internal storage. The subject technology generates a first list of metadata files on the internal storage. The subject technology generates, based on the first list of metadata files, a first set of metadata files on the internal storage. The subject technology generates a second snapshot of the table on the external volume. The subject technology generates a second list of metadata files on the external volume. The subject technology generates, based on the second list of metadata files, a second set of metadata files on the external volume. The subject technology generates a first set of data files in a different file format on the external volume.

Abstract: Embodiments of the present disclosure relate to sharing data using database roles. Database roles are generated within a database container of a provider account. Grants to a particular subset of the plurality of data objects of the database container may be assigned to each of the database roles, and each of the database roles are granted to a share object. The share object is mounted within a consumer account to generate an imported copy of each of the database roles. The imported copy of one or more of the database roles is granted to each of one or more account level roles of the consumer account. When a new object is added to a particular database role, it is immediately available for consumption by any account level roles to which the imported copy of the particular database role has been granted.

Abstract: An in-database application package and application instance for a data platform. The data platform creates an application instance of an application package having a versioned schema, creates one or more system roles for the application instance, creates a user role and an administrator role for the application instance, creates one or more objects of the application instance based on a versioned schema, and grants one or more use privileges to the one or more roles. Application instances of the application package are upgraded or patched on the data platform based on application package versions. To ensure a proper upgrade or patch, the data platform tracks versions of executing objects of application instances in a call context.

Abstract: The subject technology generates, by a compute service manager, a schema hash value for a new schema version associated with a new schema version value, the schema hash value based on determining a sum of hash values of a set of attributes of value columns, the set of attributes comprises a column identifier, and a logical type of a column. The subject technology stores a mapping of the schema hash value to the new schema version value for a table in a metadata database. The subject technology stores a new schema entry based on the schema hash value, the new schema version value, and a new column for the table in the metadata database, the metadata database storing multiple entries for different schema versions, each entry including a particular schema hash value for mapping to a corresponding schema version from the different schema versions.

Abstract: Example caching systems and methods are described. In one implementation, a method receives a query, at an execution platform, directed to data stored across a plurality of shared storage devices, the execution platform comprising one or more execution nodes, an execution node comprising a plurality of processors. The method processes the query using the one or more execution nodes of the execution platform, and in response to a determination of a change in a number of execution nodes of the execution platform, wherein the change is creating a new execution node, wherein a first subset of the plurality of processors comprises a minimal cache and a second subset of the plurality of processors comprises a cache providing faster input-output operations, reassigns processing of the query, among the changed number of execution nodes of the execution platform.