Abstract: A system for data object replication includes at least one hardware processor and at least one memory storing instructions. The instructions cause the at least one hardware processor to perform operations including decoding a replication request to obtain a data object. Object dependencies associated with the data object and a plurality of additional data objects are determined. A replication sequence of the data object and the plurality of additional data objects is determined based on the object dependencies. A replication of the data object and at least one of the plurality of additional data objects is performed according to the replication sequence. A notification of a successful completion of the replication is generated.

Abstract: In an embodiment, a data platform creates an application in a data-provider account. The application includes one or more APIs corresponding to one or more underlying code blocks. The data platform shares provider data with the application in the data-provider account, and also installs, in a data-consumer account, an application instance of the application. The application instance includes one or more APIs corresponding to the one or more APIs in the application in the data-provider account. The data platform shares consumer data with the application instance in the data-consumer account, and invokes one or more of the APIs of the application instance to execute respective associated underlying code blocks, which are not visible to the data-consumer account. The data platform also saves output of the one or more respective associated underlying code blocks locally within the data-consumer account.

Abstract: Disclosed are techniques for deduplicating files to be ingested by a database. A bloom filter may be built for each of a first set of files to be ingested into a data exchange to generate a set of bloom filters, wherein the data exchange includes a metadata storage where metadata including a list of files ingested is stored. The set of bloom filters may be stored in the metadata storage of the data exchange. In response to receiving a set of candidate files to be ingested into the data exchange, the set of bloom filters may be used to identify from within the set of candidate files, each candidate file that is duplicative of a file in the first set of files.

Abstract: A system for enforcing projection constraints on data values stored in specified variables of a shared dataset of a cloud data platform. A request is received from a first account of the cloud data platform that identifies a first operation to be performed on the shared dataset. A first set of data, including data accessed from a first variable, is accessed from the shared dataset to use in performing the first operation. A projection constraint policy attached to the first variable of the shared dataset is determined, and the projection constraint policy is further determined to be enforced based on the request. Based on the first set of data and the first operation, an output to the first request is generated.

Abstract: A constraint system enforces projection constraints on data values stored in specified columns of a shared dataset when queries are received by a database system. A projection constraint identifies that the data in a column may be restricted from being projected (e.g., presented, read, outputted) in an output to a received query, while allowing specified operations to be performed on the data and a corresponding output to be provided. For example, the projection constraint may indicate a context for a query that triggers the constraint, such as based on the user that submitted the query. Enforcing projection constraints on queries received at the database system allows for data to be shared and used anonymously by entities to perform various operations without the need to tokenize the data.

Abstract: The subject technology receives a command to commit a table in a different table format on an external volume. The subject technology generates a first snapshot of the table on internal storage. The subject technology generates a first list of metadata files on the internal storage. The subject technology generates, based on the first list of metadata files, a first set of metadata files on the internal storage. The subject technology generates a second snapshot of the table on the external volume. The subject technology generates a second list of metadata files on the external volume. The subject technology generates, based on the second list of metadata files, a second set of metadata files on the external volume. The subject technology generates a first set of data files in a different file format on the external volume.

Abstract: Described herein are techniques for improving transfer of metadata from a metadata database to a database stored in a data system, such as a data warehouse. The metadata may be written into the metadata database with a version stamp, which is monotonic increasing register value, and a partition identifier, which can be generated using attribute values of the metadata. A plurality of readers can scan the metadata database based on version stamp and partition identifier values to export the metadata to a cloud storage location. From the cloud storage location, the exported data can be auto ingested into the database, which includes a journal and snapshot table.

Abstract: A data platform creates an application in a data-provider account, where the application includes one or more application programming interfaces (APIs) corresponding to one or more underlying code blocks. The data platform shares homomorphically encrypted provider data with the application in the data-provider account. The data platform installs, in a data-consumer account, an application instance of the application. The data platform shares homomorphically encrypted consumer data with the application instance in the data-consumer account. The data platform invokes one or more of the APIs of the application instance to execute respective associated underlying code blocks, which are not visible to the data-consumer account, and which operate on the shared homomorphically encrypted provider data and the shared homomorphically encrypted consumer data. The data platform saves homomorphically encrypted output of the one or more respective associated underlying code blocks locally within the data-consumer account.

Abstract: In an embodiment, an application is created on a data-provider platform. The application includes one or more application programming interfaces (APIs) corresponding to one or more underlying code blocks. Provider data is shared with the application on the data-provider platform. An application instance of the application is installed in a trusted execution environment (TEE). The application instance includes one or more APIs corresponding to the one or more APIs in the application on the data-provider platform. Consumer data is shared with the application instance from a data-consumer platform. One or more of the APIs of the application instance are invoked to execute, on the TEE, respective associated underlying code blocks that are not visible on the TEE. The output of the one or more respective associated underlying code blocks is saved to the data-consumer platform.

Abstract: A set of affinity metrics may be determined for a set of listings, each listing of the set of listings comprising data to be shared through a data exchange, wherein the set of affinity metrics includes a set of characteristics allowing identification of a listing having one or more characteristics in the set of characteristics. For each pair of listings of the set of listings, an affinity score can be calculated, using the set of affinity metrics, and stored as part of the record in an affinity store. One or more listings of the set of listings using the affinity score between the first listing of the set of listings and the one or more listings of the set of listings can be presented.

Abstract: Provided herein are systems and methods for sharing unstructured data in stages. For example, a method includes generating a share object at an account of a data provider. The share object identifies an account of a data consumer and at least one unstructured data file shared with the account of the data consumer. The share object is configured with access privileges to the at least one unstructured data file. A notification of the share object is communicated to the account of the data consumer.

Abstract: A method of implementing sub-table replication starts with the processor detecting an update to an entitlements table. The processor performs filtering of a data table based on the update to the entitlements table. The data table including an entitlements column. The processor detects an update to the entitlements column and performs incremental replication of the data table by causing a version-based replication to be executed. Other embodiments are also described herein.

Abstract: In an embodiment, a database platform receives a query that is directed at least in part to external data in an external table stored on a data storage platform that is external to the database platform. The external table includes a plurality of partitions. The database platform identifies, from external-table metadata, a subset of the plurality of partitions of the external table as including data that potentially satisfies the query. The external-table metadata is stored by the database platform. The database platform identifies data that satisfies the query by scanning the identified subset of the partitions, and responds to the query at least in part with the identified data that satisfies the query.

Abstract: A network device communication system can configure network devices, such as a first database in a multi-tenant deployment and a second database in a private deployment, to send and receive sequence messages, such as replication data, over a channel comprising a plurality of private network nodes. The first database can create a link specifying the data share and the second database. The second database selects the link and a secure area in the private deployment is created into which data is replicated and shared with further accounts in a computationally secure and efficient manner.

Abstract: A method of implementing object tagging framework starts with the processor receiving a tag creation command including a tag name. In response to the tag creation command, the processor creates a current tag. The processor then receives an association command, the tag name and a target object identifier. The processor determines a target object associated with the target object identifier. The target object includes a tag value. The processor associates the current tag with the target object. The processor identifies a first child object of the target object. The target object and the first child object are hierarchical objects. In response to determining that the first child object is tag-unassociated, the processor associates the current tag with the first child object. In response to receiving a query including the tag name, the processor generates an output based on the tag name. Other embodiments are also described herein.

Abstract: Embodiments of the present disclosure relate to sharing database roles using hidden roles. A database role may be generated within a database container having a plurality of data objects, wherein the database role exists exclusively within the database container. A set of grants to a particular subset of the plurality of data objects of the database container may be assigned to the database role and the database role may be granted to the share object. The share object is mounted within a consumer account to generate an imported database container within the consumer account, the imported database container including an imported copy of the database role. The imported copy of the database role may be granted to each of one or more account level roles of the consumer account to share the particular subset of the plurality of data objects without creating proxy objects in the consumer account that represent the particular subset of the plurality of data objects.

Abstract: Techniques described herein can allow users to share cached results of an original query with other users while protecting sensitive information. The techniques described herein can check whether the other users have access to the underlying data queried before allowing those users to see the stored query results. That is, the system may perform privilege checks on the shared users before giving them access to the stored query results but without having to re-run the original query.

Abstract: Techniques described herein can allow users to share cached results of an original query with other users while protecting sensitive information. The techniques described herein can check whether the other users have access to the underlying data queried before allowing those users to see the stored query results. That is, the system may perform privilege checks on the shared users before giving them access to the stored query results but without having to re-run the original query.

Abstract: A data platform that implements memoizable functions for database objects. The data platform detects a first execution of a memoizable function and generates a first key based on metadata of one or more database objects operated on by the memoizable function and generates a first result for the memoizable function based on the one or more database objects. The data platform detects a second execution of the memoizable function and generates a second key based on the metadata of the one or more database objects operated on by the memoizable function. When the first key and the second key are equal, the data platform reuses the first result of the memoizable function. When the first key and second key do not match, the data platform generates a second result for the second execution of the memoizable function.

Abstract: Systems and methods for managing column hiding are provided. The systems and methods receive, from a client device, a query associated with a table. The systems and methods determine an access restriction associated with the client device. The systems and methods identify a column of the table that is restricted by the access restriction associated with the client device. In response to identifying the column of the table that is restricted by the access restriction associated with the client device, the systems and methods provide a result of the query that excludes data corresponding to the column.