Abstract: Described herein are techniques for improving transfer of metadata from a metadata database to a database stored in a data system, such as a data warehouse. The metadata may be written into the metadata database with a version stamp, which is monotonic increasing register value, and a partition identifier, which can be generated using attribute values of the metadata. A plurality of readers can scan the metadata database based on version stamp and partition identifier values to export the metadata to a cloud storage location. From the cloud storage location, the exported data can be auto ingested into the database, which includes a journal and snapshot table.

Abstract: A data dictionary generation system automatically populates and updates a data dictionary for listings offering shared data. The data listing distribution component distributes the data dictionaries to various remote deployments in a data exchange by using a global messaging framework and replication method. For example, the data listing distribution component replicates a data dictionary generated for the listing and its shared data from a source deployment to one or more destination deployments associated with various geographic regions. The data listing distribution component distributes the listing to the various remote deployments to allow for the listing, including its shared data and data dictionary, to be accessed by users within the geographic region associated with the remote deployment.

Abstract: A method includes generating at a source account of a database system user, a replication group object (RGO). The RGO includes a manifest identifying one or more data object types and one or more databases. A replication of data objects of the one or more data object types is performed from the one or more databases to a target account of the database system user. The replication of the data objects is performed as a unit. The RGO is replicated as a failover group object (FGO) within the target account of the database system user. A network failure is detected at the source account of the database system user. The target account of the database system user is promoted as a new source account based on the detecting of the network failure.

Abstract: Various embodiments provide for replicating a share across deployments of a data platform, where the share can be on a source deployment and the share can be replicated on one or more target deployments, and where the share is replicated with one or more database objects of the source deployment associated with the share. Some embodiments analyze the share to be replicated and, based on the analysis, determine one or more database objects that would be replicated to the one or more target deployments to enable a replica of the share on the one or more target deployments.

Abstract: A data platform creates an application in a data-provider account, where the application includes one or more application programming interfaces (APIs) corresponding to one or more underlying code blocks. The data platform shares homomorphically encrypted provider data with the application in the data-provider account. The data platform installs, in a data-consumer account, an application instance of the application. The data platform shares homomorphically encrypted consumer data with the application instance in the data-consumer account. The data platform invokes one or more of the APIs of the application instance to execute respective associated underlying code blocks, which are not visible to the data-consumer account, and which operate on the shared homomorphically encrypted provider data and the shared homomorphically encrypted consumer data. The data platform saves homomorphically encrypted output of the one or more respective associated underlying code blocks locally within the data-consumer account.

Abstract: A method for table data processing includes retrieving, by at least one hardware processor, first table data associated with a first table version from a plurality of partitions of a storage device. The method further includes detecting, by the at least one hardware processor, an executed transaction causing a modification to the first table data resulting in a second table data associated with a second table version. The second table data is modified to include partition metadata. The partition metadata includes table version information with a mapping of the first table version and the second table version to a plurality of transactions associated with generating the first table data and the second table data.

Abstract: A versioned schema of a data platform. A process of maintaining a call stack of executing objects of an application package having a versioned schema includes calling, by a first procedure executed by one or more processors, a second procedure of a versioned application instance, and determining, by the first procedure, a version of the second procedure based on a call context. In response to determining that the version of the second procedure is not in the call context, the first procedure determines a current version of the versioned application package adds the current version to the call context as the version of the second procedure.

Abstract: Provided herein are systems and methods for configuring replication group objects. A system includes at least one hardware processor coupled to memory and configured to decode a replication request received from a client device of a data provider. The replication request indicates a plurality of account objects for replication. The plurality of account objects associated with a corresponding plurality of account object types. A replication group object is generated based on the replication request. The replication group object includes a manifest. The manifest lists the plurality of account objects. The replication of the plurality of account objects is performed from a source account of the data provider into at least one target account. The replication is based on the manifest of the replication group object.

Abstract: Techniques for creating, sharing, and using bundles (also referred to as packages) in a multi-tenant database are described herein. A bundle is a schema object with associated hidden schemas. A bundle can be created by a provider user and can be shared with a plurality of consumer users. The bundle can be used to enable code sharing and distribution without losing control while maintaining security protocols.

Abstract: Row-level security (RLS) may provide fine-grained access control based on flexible, user-defined access policies to databases, tables, objects, and other data structures. A RLS policy may be an entity or object that defines rules for row access. A RLS policy may be decoupled or independent from any specific table. This allows more robust and flexible control. A RLS policy may then be attached to one or more tables. The RLS policy may include a Boolean-valued expression.

Abstract: Various embodiments provide for tag-based application of a masking policy, which can be used in connection with a data platform. In particular, various embodiments enable enforcement of one or more masking policies against an entity (e.g., object) of a data platform, such as a database, a table, a row, or a column, based on one or more tags associated with the entity.

Abstract: Techniques for configuring event tables include retrieving, by at least one hardware processor of a computing node, log data at a first process of the computing node. The log data is associated with a function executing at a second process of the computing node. The log data is revised using a table stage to generate revised log data. The table stage is configured as a dedicated storage location of the first process. The revising includes a data enrichment process based on metadata associated with execution of the function at the second process. The revised log data is ingested into an event table.

Abstract: A system for sub-table replication within a materialized view of a source data table. A materialized view of a source data table is generated based on a query, the source data table having a plurality of data table rows associated with a plurality of client identifiers and including an entitlements column and data content, the entitlements column including a plurality of provider account identifiers and a plurality of consumer account identifiers. An update to an entitlements table of the source data table is detected where the entitlements table includes a plurality of entitlements table rows and where the entitlements table rows are associated with the client identifiers, the consumer account identifiers, and the provider account identifiers. An update to the source data table based on the first update to the entitlements table is detected, and an incremental replication of the materialized view of the source data table is performed.

Abstract: Techniques for database replication for a remote deployment include a method performed by a data platform executing instructions on at least one hardware processor. The method includes detecting receipt of a database replication request associated with a data consumer. The database replication request is requesting availability of an instance of a primary deployment database at a remote deployment of a data platform. In response to detecting the receipt of the database replication request, an account of a data provider at the remote deployment of the data platform is provisioned with a set of replication-preparation objects associated with the primary deployment database. A database refresh operation is invoked, causing database data be pushed from the primary deployment database to a database shell in the account of the data provider at the remote deployment, to create the instance of the primary deployment database.

Abstract: Described herein are techniques for improving transfer of metadata from a metadata database to a database stored in a data system, such as a data warehouse. The metadata may be written into the metadata database with a version stamp, which is monotonic increasing register value, and a partition identifier, which can be generated using attribute values of the metadata. A plurality of readers can scan the metadata database based on version stamp and partition identifier values to export the metadata to a cloud storage location. From the cloud storage location, the exported data can be auto ingested into the database, which includes a journal and snapshot table.

Abstract: Various embodiments provide for replicating a share across deployments of a data platform, where the share can be on a source deployment and the share can be replicated on one or more target deployments, and where the share is replicated with one or more database objects of the source deployment associated with the share. Some embodiments analyze the share to be replicated and, based on the analysis, determine one or more database objects that would be replicated to the one or more target deployments to enable a replica of the share on the one or more target deployments.

Abstract: The embodiments described herein provide means for replicating external stages between deployments of e.g., a cloud data lake using a modified storage integration. The modified storage integration may be defined with a set of storage locations, wherein the storage integration comprises a base URL for each of the set of storage locations and wherein each storage location identifies a remote deployment where a cloud platform is hosted and a geographic region of the remote deployment. An external stage object may be bound to the storage integration, wherein the external stage object facilitates a data loading operation that is currently in progress on the first storage location. In response to detecting an outage at the first storage location, the data loading operation that was in progress on the first storage location may be continued at the second storage location using the storage integration.

Abstract: Aspects of the present disclosure address systems, methods, and devices for tracking object dependencies in a cloud database system. An object dependency created between a referencing object and a referenced object is detected. Based on detecting the object dependency, a dependency record is generated. The dependency record includes dependency information describing the object dependency between the reference object and the referenced object. The dependency record is stored in a database of dependency records.

Abstract: Affinity-based listing recommendations are created and used in a public data exchange. Listings can be evaluated against one another for affinity or similarity such that users working with a particular dataset can be presented with other datasets that share an affinity. Affinity can be determined from both the dataset metadata as well as information from the dataset content. Calculation of affinity scores can be pre-computed and stored, in advance of use, or determined on-the-fly. Presentation of most-similar listings can be deterministic, can contain randomization, can employ time-decay, can be weighted, and can make use of a tiered-sum approach.

Abstract: Embodiments of the present disclosure relate to sharing database roles using hidden roles. A database role may be generated within a database container having a plurality of data objects, wherein the database role exists exclusively within the database container. A set of grants to a particular subset of the plurality of data objects of the database container may be assigned to the database role and the database role may be granted to the share object. The share object is mounted within a consumer account to generate an imported database container within the consumer account, the imported database container including an imported copy of the database role. The imported copy of the database role may be granted to each of one or more account level roles of the consumer account to share the particular subset of the plurality of data objects without creating proxy objects in the consumer account that represent the particular subset of the plurality of data objects.