Abstract: A method includes generating at a source account of a database system user, a replication group object (RGO). The RGO includes a manifest identifying one or more data object types and one or more databases. A replication of data objects of the one or more data object types is performed from the one or more databases to a target account of the database system user. The replication of the data objects is performed as a unit. The RGO is replicated as a failover group object (FGO) within the target account of the database system user. A network failure is detected at the source account of the database system user. The target account of the database system user is promoted as a new source account based on the detecting of the network failure.

Abstract: This disclosure provides methods and techniques of data replication involving cross replication group (RG) references. The present disclosure avoids automatic replication failing when an entity in an RG refers to another entity external to the RG. The entity to be replicated within the RG is referred to as the “referring entity,” and the entity as the dangling reference is referred to as the “referred entity.” Although the referring and referred entities are not replicated together in a replication operation, the referred entity may have already been replicated to the target account in another replication operation on a different replication group. In such cases, the data replication procedure may, according to aspects of the present disclosure, check if the referred entity has already been replicated, and if so, proceed to replicate the referring entity without fail, and link the referring and referred entities to enable normal functioning of the referring entity.

Abstract: The subject technology provides techniques for enabling hidden database objects, which in an example are utilized for testing and verifying new database objects against existing workloads. Hidden database objects are a mechanism for bridging that gap by running user workloads on user data in advance of exposing the feature to users. This mechanism allows placing a database object as a hidden object nested beneath a user's visible object (e.g., table, column, view, and the like). Hidden database objects can be used to verify the functionality, parity, performance, and correctness of new unreleased features.

Abstract: The subject technology determines a set of share objects in a replication group. The subject technology, for each object from the set of share objects, determines a set of permissions for a set of objects of databases that has been granted to each share object from the replication group. The subject technology, for each permission from the set of permissions, determines a set of dependencies, the set of dependencies comprising a second set of objects. The subject technology generates a graph structure based at least in part on the set of share objects, the set of permissions, and the set of dependencies. The subject technology traverses at least a portion of the graph structure to determine a final set of dependencies for replication. The subject technology performs a replication process based at least in part on the final set of dependencies.

Abstract: The subject technology provides techniques for enabling hidden database objects, which in an example are utilized for testing and verifying new database objects against existing workloads. Hidden database objects are a mechanism for bridging that gap by running user workloads on user data in advance of exposing the feature to users. This mechanism allows placing a database object as a hidden object nested beneath a user's visible object (e.g., table, column, view, and the like). Hidden database objects can be used to verify the functionality, parity, performance, and correctness of new unreleased features.

Abstract: The subject technology provides embodiments for supporting a unified table which may be a managed table or an unmanaged table. Managed tables are those where the subject technology manages the metastore/catalog for the table, whereas unmanaged tables are tables where an external catalog controls the table and the subject technology integrates with that catalog to work with the table, but does not assume control of the table.

Abstract: Different database deployments, or other data system deployments, may want to communicate with each other without sacrificing security or control. To this end, embodiments of the present disclosure may provide secure message exchange techniques for a source and/or target deployment. Configurable rule sets may be stored in the deployments; the rule sets may define what messages may be communicated between deployments. The deployments may implement a selective filtering scheme in one or more stages based on the rule sets to filter outgoing and/or incoming messages.

Abstract: Described herein are techniques for improving disaster recovery, in particular disaster recovery pertaining to data transfer requests. The data transfer request can be received by each of multiple deployments; however, only a primary deployment can process the request. The data transferred by the primary deployment may be replicated in the secondary deployments. In response to a failover event, one of the secondary deployments can be designated as the new primary development and continue the data transfer based on the data transfer request and the replication information received from the old primary deployment prior to the failover.

Abstract: A system for sub-table replication within a materialized view of a source data table. A materialized view of a source data table is generated based on a query, the source data table having a plurality of data table rows associated with a plurality of client identifiers and including an entitlements column and data content, the entitlements column including a plurality of provider account identifiers and a plurality of consumer account identifiers. An update to an entitlements table of the source data table is detected where the entitlements table includes a plurality of entitlements table rows and where the entitlements table rows are associated with the client identifiers, the consumer account identifiers, and the provider account identifiers. An update to the source data table based on the first update to the entitlements table is detected, and an incremental replication of the materialized view of the source data table is performed.

Abstract: This disclosure provides methods and techniques of data replication involving cross replication group (RG) references. The present disclosure avoids automatic replication failing when an entity in an RG refers to another entity external to the RG. The entity to be replicated within the RG is referred to as the “referring entity,” and the entity as the dangling reference is referred to as the “referred entity.” Although the referring and referred entities are not replicated together in a replication operation, the referred entity may have already been replicated to the target account in another replication operation on a different replication group. In such cases, the data replication procedure may, according to aspects of the present disclosure, check if the referred entity has already been replicated, and if so, proceed to replicate the referring entity without fail, and link the referring and referred entities to enable normal functioning of the referring entity.

Abstract: Techniques for executing show commands are described herein. A plurality of navigation steps is utilized, each navigation step corresponding to a different layer in a database structure and each navigation step including an operator to fetch items from a metadata database up to respective bounded limits. Dependency information is also fetched for objects of the specified object type in the show command. After a set of objects from the last layer are processed, memory for the navigation steps is flushed and the next set of objects are processed.

Abstract: Various embodiments provide for replicating a share across deployments of a data platform, where the share can be on a source deployment and the share can be replicated on one or more target deployments, and where the share is replicated with one or more database objects of the source deployment associated with the share. Some embodiments analyze the share to be replicated and, based on the analysis, determine one or more database objects that would be replicated to the one or more target deployments to enable a replica of the share on the one or more target deployments.

Abstract: Different database deployments, or other data system deployments, may want to communicate with each other without sacrificing security or control. To this end, embodiments of the present disclosure may provide secure message exchange techniques for a source and/or target deployment. Configurable rule sets may be stored in the deployments; the rule sets may define what messages may be communicated between deployments. The deployments may implement a selective filtering scheme in one or more stages based on the rule sets to filter outgoing and/or incoming messages.

Abstract: A data transfer request can be received by each of multiple deployments; however, only a primary deployment can process the request. The data transferred by the primary deployment may be replicated in the secondary deployments. In response to a failover event, one of the secondary deployments can be designated as the new primary development and continue the data transfer based on the data transfer request and the replication information received from the old primary deployment prior to the failover.

Abstract: A system for sub-table replication within a materialized view of a source data table. A materialized view of a source data table is generated based on a query, the source data table having a plurality of data table rows associated with a plurality of client identifiers and including an entitlements column and data content, the entitlements column including a plurality of provider account identifiers and a plurality of consumer account identifiers. An update to an entitlements table of the source data table is detected where the entitlements table includes a plurality of entitlements table rows and where the entitlements table rows are associated with the client identifiers, the consumer account identifiers, and the provider account identifiers. An update to the source data table based on the first update to the entitlements table is detected, and an incremental replication of the materialized view of the source data table is performed.

Abstract: As described herein, a N-Gram index may be created and the search may be conducted using the index, which will lead to faster search results. The N-Gram index may also include partial N-Gram components to capture more relevant data. Moreover, as described herein, the search may also take into account recent log data that has not yet been indexed. Techniques for building an index store using log data and efficiently searching the index store and log data to process search requests are described herein.

Abstract: Techniques for executing show commands are described herein. A plurality of navigation steps is utilized, each navigation step corresponding to a different layer in a database structure and each navigation step including an operator to fetch items from a metadata database up to respective bounded limits. Dependency information is also fetched for objects of the specified object type in the show command. After a set of objects from the last layer are processed, memory for the navigation steps is flushed and the next set of objects are processed.

Abstract: The subject technology provides embodiments for supporting a unified table which may be a managed table or an unmanaged table. Managed tables are those where the subject technology manages the metastore/catalog for the table, whereas unmanaged tables are tables where an external catalog controls the table and the subject technology integrates with that catalog to work with the table, but does not assume control of the table.

Abstract: Described herein are techniques for compact representation of table columns via templatization. Column templates can model the collection of columns in a table as a logical entity. The logical entity can be split into two objects. A first object can contain a subset of column attributes (e.g., fields) that can be shared with other tables, such as name and data type. A second object can contain another subset of column attributes that are unique to a table, such as time stamps and created-on information.

Abstract: As described herein, a N-Gram index may be created and the search may be conducted using the index, which will lead to faster search results. The N-Gram index may also include partial N-Gram components to capture more relevant data. Moreover, as described herein, the search may also take into account recent log data that has not yet been indexed. Techniques for building an index store using log data and efficiently searching the index store and log data to process search requests are described herein.