Abstract: A method provides deterministic row ordering for query result sets in a database system. A framework receives a query referencing a database table and generating a row index column in response. The framework generates the row index column that involves determining an order associated with source files underlying the referenced table and assigns each table row a unique index value based on the file order and row position within the files. The framework executes the received query to produce a result set ordered by the generated row index column. The framework determines row order based on analyzing underlying data file metadata and facilitates returning the result set with consistent, reliable ordering, enabling stable positional-based access semantics when querying distributed database tables.

Abstract: Examples provide a method for performing dynamic pivot queries employing PANDA-compatible semantics inside a cloud data platform workload. A SQL engine receives a pivot query specifying a pivot column for dynamically determining pivot values in a dynamic pivot operation. The SQL engine identifies, in the pivot query, a keyword or a subquery indicating dynamic determination of the pivot values. Based on identifying the keyword or the subquery indicating the dynamic determination of the pivot values, the SQL engine determines distinct pivot values for the pivot column by evaluating the keyword or executing the subquery. The evaluating of the keyword includes issuing a query to a data store to obtain the distinct pivot values for the pivot column. The SQL engine performs the dynamic pivot operation using the distinct pivot values for the pivot column.

Abstract: A method provides deterministic row ordering for query result sets in a database system. A framework receives a query referencing a database table and generating a row index column in response. The framework generates the row index column that involves determining an order associated with source files underlying the referenced table and assigns each table row a unique index value based on the file order and row position within the files. The framework executes the received query to produce a result set ordered by the generated row index column. The framework determines row order based on analyzing underlying data file metadata and facilitates returning the result set with consistent, reliable ordering, enabling stable positional-based access semantics when querying distributed database tables.

Abstract: A method includes receiving instructions to perform order-dependent DataFrame operation on data. In response to receiving the instructions, the framework analyzes the instructions to identify the order-dependent DataFrame operation, and generates an executable query corresponding to the identified order-dependent DataFrame operation. The framework executes the generated executable query on the data stored in the first database in the cloud data platform, creates a row position column that generates for each row a unique integer identifier, reproducible across multiple sessions and/or queries for the same underlying data to make the data accessible via positional indexing, and assigns each row of the first database a unique row index value based on the row position column and the row position column order.

Abstract: An external function system can be implemented on a database to perform processing on one or more external network services. The external function system can comprise a particular external function for a particular external service, an outbound serializer function, and an inbound serializer function that are linked with the particular external function. The outbound serializer function can be configured to transform the data of a query from a database format to a different format of the particular external network service. The inbound deserializer function can be configured to receive data returned from the external service and transform the data to the format of the database.

Abstract: Methods, systems, and computer programs are presented for a policy to enable customers to create and apply allowed items and restricted items provided by third-party managers. User code is received by a cloud data platform, the code for performing an operation. The cloud data platform receives a policy including allowed items and restricted items. The cloud data platform compares the allowed and restricted items to a specification, and determines a difference based on the comparison. The cloud data platform determines whether to permit the operation based at least in part on the differences.

Abstract: Multiple execution environments are established for processing database statements with diverse package dependencies. A database system receives a user application and identifies, via hardware processors, multiple package dependencies required by the application, including at least a first and a second package dependency. Upon receiving a database statement from a user that invokes these dependencies, the system retrieves the necessary packages from a repository. It then creates a first execution environment incorporating the first package dependency and a second execution environment with the second package dependency. Utilizing the multiple execution environments, the system generates results data in response to the database statement, effectively managing and executing user-defined functions that rely on different versions or types of package dependencies within a unified database framework.

Abstract: A database dependency resolver system can identify different dependencies of a user application and integrate the identified dependencies in different execution environments of a distributed database system. The different execution environments can manage different versions of a given programming language, or other types of computational architectures (e.g., different CPU types). A database user can provide a database statement (e.g., query) that activates the different dependencies in the different environments to generate results data.

Abstract: An external function system can be implemented on a database to perform processing on one or more external network services. The external function system can comprise a particular external function for a particular external service, an outbound serializer function, and an inbound serializer function that are linked with the particular external function. The outbound serializer function can be configured to transform the data of a query from a database format to a different format of the particular external network service. The inbound deserializer function can be configured to receive data returned from the external service and transform the data to the format of the database.

Abstract: An external function system can be implemented on a database to perform processing on one or more external network services. The external function system can comprise a particular external function for a particular external service, an outbound serializer function, and an inbound serializer function that are linked with the particular external function. The outbound serializer function can be configured to transform the data of a query from a database format to a different format of the particular external network service. The inbound deserializer function can be configured to receive data returned from the external service and transform the data to the format of the database.

Abstract: Methods, systems, and computer programs are presented for a packages policy object to enable customers to create and apply packages policies provided by third-party package managers. A user-defined function (UDF) is received by a cloud data platform. The UDF includes code related to at least one operation to be performed. The cloud data platform receives a package policy including at least one allowlist and at least one blocklist and compares the at least one allowlist and the at least one blocklist to the at least one package specification. The cloud data platform computes a difference set based on the comparison. The difference set includes metadata associated with one or more packages to remove. The cloud data platform determines whether the UDF is permitted based at least in part on the difference set.

Abstract: A database dependency resolver system can identify different dependencies of a user application and integrate the identified dependencies in different execution environments of a distributed database system. The different execution environments can manage different versions of a given programming language, or other types of computational architectures (e.g., different CPU types). A database user can provide a database statement (e.g., query) that activates the different dependencies in the different environments to generate results data.

Abstract: A query referencing an external table function provided by a remote software component is received. Requests to execute the external table function on input data are sent to a proxy service. A first request includes a batch of input rows from the input data. A first response to the first request received from the proxy service includes a first portion of result data and a pagination token. The pagination token indicates that at least a second portion of the result data corresponding to the first batch of input rows is to be obtained from the remote software component. Based on the pagination token, a second request is sent to obtain the second portion of the result data. One or more responses are received from the proxy service that comprise at least the second portion of the result data. The result data is processed according to the query.

Abstract: An external function system can be implemented on a database to perform processing on one or more external network services. The external function system can comprise a particular external function for a particular external service, an outbound serializer function, and an inbound serializer function that are linked with the particular external function. The outbound serializer function can be configured to transform the data of a query from a database format to a different format of the particular external network service. The inbound deserializer function can be configured to receive data returned from the external service and transform the data to the format of the database.

Abstract: A query referencing an external table function provided by a remote software component is received. Requests to execute the external table function on input data are sent to a proxy service. A first request includes a batch of input rows from the input data. A first response to the first request received from the proxy service includes a first portion of result data and a pagination token. The pagination token indicates that at least a second portion of the result data corresponding to the first batch of input rows is to be obtained from the remote software component. Based on the pagination token, a second request is sent to obtain the second portion of the result data. One or more responses are received from the proxy service that comprise at least the second portion of the result data. The result data is processed according to the query.

Abstract: A query referencing an external table function provided by a remote software component is received. Requests to execute the external table function on input data are sent to a proxy service. A first request includes a batch of input rows from the input data. A first response to the first request received from the proxy service includes a first portion of result data and a pagination token. The pagination token indicates that at least a second portion of the result data corresponding to the first batch of input rows is to be obtained from the remote software component. Based on the pagination token, a second request is sent to obtain the second portion of the result data. One or more responses are received from the proxy service that comprise at least the second portion of the result data. The result data is processed according to the query.

Abstract: A query referencing an external table function provided by a remote software component is received. Requests to execute the external table function on input data are sent to a proxy service. A first request includes a batch of input rows from the input data. A first response to the first request received from the proxy service includes a first portion of result data and a pagination token. The pagination token indicates that at least a second portion of the result data corresponding to the first batch of input rows is to be obtained from the remote software component. Based on the pagination token, a second request is sent to obtain the second portion of the result data. One or more responses are received from the proxy service that comprise at least the second portion of the result data. The result data is processed according to the query.

Abstract: An external function system can be implemented on a database to perform processing on one or more external network services. The external function system can comprise a particular external function for a particular external service, an outbound serializer function, and an inbound serializer function that are linked with the particular external function. The outbound serializer function can be configured to transform the data of a query from a database format to a different format of the particular external network service. The inbound deserializer function can be configured to receive data returned from the external service and transform the data to the format of the database.

Abstract: A query referencing an external table function provided by a remote software component is received. Requests to execute the external table function on input data are sent to a proxy service. A first request includes a batch of input rows from the input data. A first response to the first request received from the proxy service includes a first portion of result data and a pagination token. The pagination token indicates that at least a second portion of the result data corresponding to the first batch of input rows is to be obtained from the remote software component. Based on the pagination token, a second request is sent to obtain the second portion of the result data. One or more responses are received from the proxy service that comprise at least the second portion of the result data. The result data is processed according to the query.

Abstract: A query referencing an external table function provided by a remote software component is received. Requests to execute the external table function on input data are sent to a proxy service. A first request includes a batch of input rows from the input data. A first response to the first request received from the proxy service includes a first portion of result data and a pagination token. The pagination token indicates that at least a second portion of the result data corresponding to the first batch of input rows is to be obtained from the remote software component. Based on the pagination token, a second request is sent to obtain the second portion of the result data. One or more responses are received from the proxy service that comprise at least the second portion of the result data. The result data is processed according to the query.