Abstract: The subject technology identifies a set of functions in a set of files corresponding to a library. The subject technology, for each function, registers the function as a user defined function (UDF) based on a set of input parameters utilized by the function and a type of parameter of each of the input parameters. The subject technology provides access to each registered function in a different application.

Abstract: A system for improving task scheduling on a cloud data platform is provided. A task is received, from a user of a cloud data platform, for execution on a dataset of a cloud data platform using a plurality of resources. A task graph is generated, and metadata related to the dataset is accessed for use in execution of the task. A predicted resource profile is generated by applying a first machine learning scheme to the task graph and the metadata of the dataset. Assignment data is generated to execute processes of the task on the plurality of resources. The assignment data generated by applying a second machine learning scheme to current state data of a current computational state of the plurality of resources and the predicted resource profile generated by the first machine learning scheme.

Abstract: Provided herein are systems and methods for a database object (e.g., materialized table) configuration including scheduling refreshes of the materialized table. For example, a method includes determining a dependency graph for a first MT. The dependency graph comprises a second MT from which the first MT depends. The first MT includes a query on one or more base tables and a lag duration value. The lag duration value indicates a maximum time period that a result of a prior refresh of the query can lag behind a current time instance. A tick period is selected for a set of ticks based on the lag duration value. The set of ticks corresponds to a set of aligned time instances. Refresh operations are scheduled for the first and second MTs at corresponding time instances from the set of aligned time instances. The corresponding time instances are separated by the tick period.

Abstract: A differentially private security system communicatively coupled to a database storing restricted data receives a database query from a client. The database query includes a relation specifying a set of data in the database upon which to perform the query and privacy parameters associated with the query. The differentially private security system determines a worst-case privacy spend for the query based on the privacy parameters and the relation. The differentially private security system performs the query upon the set of data specified by the relation and decrements the determined worst-case privacy spend from a privacy budget associated with the client. The differentially private security system records the worst-case privacy spend and the query at a log and determines a privacy budget refund based on queries recorded in the log. The differentially private security system applies the determined privacy budget refund to the privacy budget associated with the client.

Abstract: Generating an anonymized view for a data set is described. An example method can include receiving data from a data set, wherein the data is organized in a plurality of columns. The method may also include generating a plurality of generalizations of the data. The method may also further include selecting a generalization from the plurality of generalizations using an information loss scoring function based on at least a generalization information loss. Additionally, the method may also include generating an anonymized view of the data set from the selected generalization.

Abstract: A versioned schema of a data platform. A process of maintaining a call stack of executing objects of an application package having a versioned schema includes calling, by a first procedure executed by one or more processors, a second procedure of a versioned application instance, and determining, by the first procedure, a version of the second procedure based on a call context. In response to determining that the version of the second procedure is not in the call context, the first procedure determines a current version of the versioned application package adds the current version to the call context as the version of the second procedure.

Abstract: Example systems and methods for cloning catalog objects are described. In one implementation, a method includes creating a copy of a catalog object without copying a data associated with the catalog object by only coping metadata associated with the object. The method further includes modifying, by one or more processors, the data associated with the catalog object independently of the copy of the catalog object.

Abstract: The subject technology receives, during a query compilation process, a query directed to a set of source tables. The subject technology performs, during the query compilation process, a modification of the query for adjusting at least one pruning operation. The subject technology determines, during a pruning process of a second query, the second query directed to a set of files in a database system and including a set of pruning operations on the set of files, whether to perform a pruning cutoff on the set of pruning operations, the pruning process performing a depth first search of a pruner tree structure, the set of files comprising a set of micro-partitions. The subject technology performs the pruning cutoff based on the determining, the pruning cutoff ceasing at least one pruning operation from the set of pruning operations.

Abstract: In one aspect, a method includes forming, at a database platform, a file stage in a user account on the database platform, the user account being associated with a client, forming, at the database platform, a hidden stage directory table and a data pipeline for the file stage, and binding the data pipeline to the file stage. The method also includes receiving a notification of an event of the file stage. The method also includes identifying the data pipeline corresponding to the file stage. The method also includes updating the hidden stage directory table of the identified data pipeline.

Abstract: A proxy revocation service provides a reliable service for performing revocation checks. The proxy revocation service queries public certificate authorities for the revocation status of a set of digital certificates and maintains a database of the revocation statuses. The proxy revocation service provides a singular endpoint that is Application Protocol Interface (API) accessible to web clients. Web clients communicate with the proxy revocation service through use of API message to perform revocation checks, rather than communicating with the public certificate authorities using an online certificate status protocol (OCSP). Use of the proxy revocation service provides both a reliable service for performing revocation checks as well as shifts the complexity away from the web clients.

Abstract: Example caching systems and methods are described. In one implementation, a method identifies multiple files used to process a query and distributes each of the multiple files to a particular execution node to execute the query. Each execution node determines whether the distributed file is stored in the execution node's cache. If the execution node determines that the file is stored in the cache, it processes the query using the cached file. If the file is not stored in the cache, the execution node retrieves the file from a remote storage device, stores the file in the execution node's cache, and processes the query using the file.

Abstract: Embodiments of the present disclosure may provide a data clean room allowing secure data analysis across multiple accounts, without the use of third parties. Each account may be associated with a different company or party. The data clean room may provide security functions to safeguard sensitive information. For example, the data clean room may restrict access to data in other accounts. The data clean room may also restrict which data may be used in the analysis and may restrict the output. The overlap data may be anonymized to prevent sensitive information from being revealed.

Abstract: Techniques for schema mismatch detection and evolution are described. When data is being uploaded into a source table, schema of the data to be uploaded can be compared with the schema for the source table. If a schema mismatch is detected, the schema of the source table can be modified, and the upload can be continued without data loss.

Abstract: A system or persistent table may be generated storing changelog information of a primary base table. The system table may then be used to create streams of relevant information. In some examples, the streams may read from the system table for information past a retention period of the primary table while reading from the primary table information in the retention period.

Abstract: Provided herein are systems and methods for configuring replication group objects. A system includes at least one hardware processor coupled to memory and configured to decode a replication request received from a client device of a data provider. The replication request indicates a plurality of account objects for replication. The plurality of account objects associated with a corresponding plurality of account object types. A replication group object is generated based on the replication request. The replication group object includes a manifest. The manifest lists the plurality of account objects. The replication of the plurality of account objects is performed from a source account of the data provider into at least one target account. The replication is based on the manifest of the replication group object.

Abstract: Provided herein are systems and methods for processing a script file generated using a declarative engine for workloads (DEW). A system includes at least one hardware processor coupled to a memory and configured to retrieve a script file. The script file specifies a plurality of tasks forming a plan and an order of execution for the plurality of tasks. The script file is compiled to generate compiled code. The compiled code is executed to generate an in-memory representation of the plan. The in-memory representation of the plan is converted into a graph such as a directed acyclic graph (DAG). The graph includes the plurality of tasks. The plurality of tasks in the graph are executed according to the order of execution.

Abstract: A cloud extension agent can be provided on a customer premise for interfacing, via an outbound secure connection, cloud based services.

Abstract: A data platform for managing an application as a first-class database object. The data platform includes at least one processor and a memory storing instructions that cause the at least one processor to perform operations including detecting a data request from a browser for a data object located on the data platform, executing a stored procedure, the stored procedure containing instructions that cause the at least one processor to perform additional operations including instantiating a User Defined Function (UDF) server, an application engine, and the application within a security context of the data platform based on a security policy determined by an owner of the data object. The data platform then communicates with the browser using the application engine as a proxy server.

Abstract: Sharing data in a data exchange across multiple cloud computing platforms and/or cloud computing platform regions is described. An example method can include generating a consumer account corresponding to a first cloud entity and receiving, by the first cloud entity, a copy of a data set from a provider account corresponding to a second cloud computing entity, wherein the first cloud computing entity and the second cloud computing entity represent different regions of a cloud computing platform. The method may also include accessing, by the consumer account, the copy of the data set.

Abstract: Different database deployments, or other data system deployments, may want to communicate with each other without sacrificing security or control. To this end, embodiments of the present disclosure may provide secure message exchange techniques for a source and/or target deployment. Configurable rule sets may be stored in the deployments; the rule sets may define what messages may be communicated between deployments. The deployments may implement a selective filtering scheme in one or more stages based on the rule sets to filter outgoing and/or incoming messages.