Abstract: A database dependency resolver system can identify different dependencies of a user application and integrate the identified dependencies in different execution environments of a distributed database system. The different execution environments can manage different versions of a given programming language, or other types of computational architectures (e.g., different CPU types). A database user can provide a database statement (e.g., query) that activates the different dependencies in the different environments to generate results data.

Abstract: Methods, systems, and computer programs are presented for enabling any sandboxed user-defined function code to securely access the Internet via a cloud data platform. A remote procedure call is received by a cloud data platform from a user-defined function (UDF) executing within a sandbox process. The UDF includes code related to at least one operation to be performed. The cloud data platform provides an overlay network to establish a secure egress path for UDF external access. The cloud data platform enables the UDF executing in the sandbox process to initiate a network call.

Abstract: An external function system can be implemented on a database to perform processing on one or more external network services. The external function system can comprise a particular external function for a particular external service, an outbound serializer function, and an inbound serializer function that are linked with the particular external function. The outbound serializer function can be configured to transform the data of a query from a database format to a different format of the particular external network service. The inbound deserializer function can be configured to receive data returned from the external service and transform the data to the format of the database.

Abstract: An external function system can be implemented on a database to perform processing on one or more external network services. The external function system can comprise a particular external function for a particular external service, an outbound serializer function, and an inbound serializer function that are linked with the particular external function. The outbound serializer function can be configured to transform the data of a query from a database format to a different format of the particular external network service. The inbound deserializer function can be configured to receive data returned from the external service and transform the data to the format of the database.

Abstract: Systems and methods for caching file system collections atomically. The systems and methods perform operations comprising: receiving a request to access a collection of data comprising a plurality of files stored on a storage device; in response to receiving the request, transferring the collection of data from the storage device to a first subfolder in a cache associated with the storage device; generating a lock file comprising a reference to the first subfolder in the cache; and atomically controlling access to the collection of data in the first subfolder and removal of the collection of data in the first subfolder from the cache via the lock file.

Abstract: A database dependency resolver system can identify different dependencies of a user application and integrate the identified dependencies in different execution environments of a distributed database system. The different execution environments can manage different versions of a given programming language, or other types of computational architectures (e.g., different CPU types). A database user can provide a database statement (e.g., query) that activates the different dependencies in the different environments to generate results data.

Abstract: A query referencing an external table function provided by a remote software component is received. Requests to execute the external table function on input data are sent to a proxy service. A first request includes a batch of input rows from the input data. A first response to the first request received from the proxy service includes a first portion of result data and a pagination token. The pagination token indicates that at least a second portion of the result data corresponding to the first batch of input rows is to be obtained from the remote software component. Based on the pagination token, a second request is sent to obtain the second portion of the result data. One or more responses are received from the proxy service that comprise at least the second portion of the result data. The result data is processed according to the query.

Abstract: An external function system can be implemented on a database to perform processing on one or more external network services. The external function system can comprise a particular external function for a particular external service, an outbound serializer function, and an inbound serializer function that are linked with the particular external function. The outbound serializer function can be configured to transform the data of a query from a database format to a different format of the particular external network service. The inbound deserializer function can be configured to receive data returned from the external service and transform the data to the format of the database.

Abstract: A query referencing an external table function provided by a remote software component is received. Requests to execute the external table function on input data are sent to a proxy service. A first request includes a batch of input rows from the input data. A first response to the first request received from the proxy service includes a first portion of result data and a pagination token. The pagination token indicates that at least a second portion of the result data corresponding to the first batch of input rows is to be obtained from the remote software component. Based on the pagination token, a second request is sent to obtain the second portion of the result data. One or more responses are received from the proxy service that comprise at least the second portion of the result data. The result data is processed according to the query.

Abstract: A query referencing an external table function provided by a remote software component is received. Requests to execute the external table function on input data are sent to a proxy service. A first request includes a batch of input rows from the input data. A first response to the first request received from the proxy service includes a first portion of result data and a pagination token. The pagination token indicates that at least a second portion of the result data corresponding to the first batch of input rows is to be obtained from the remote software component. Based on the pagination token, a second request is sent to obtain the second portion of the result data. One or more responses are received from the proxy service that comprise at least the second portion of the result data. The result data is processed according to the query.

Abstract: A query referencing an external table function provided by a remote software component is received. Requests to execute the external table function on input data are sent to a proxy service. A first request includes a batch of input rows from the input data. A first response to the first request received from the proxy service includes a first portion of result data and a pagination token. The pagination token indicates that at least a second portion of the result data corresponding to the first batch of input rows is to be obtained from the remote software component. Based on the pagination token, a second request is sent to obtain the second portion of the result data. One or more responses are received from the proxy service that comprise at least the second portion of the result data. The result data is processed according to the query.

Abstract: An external function system can be implemented on a database to perform processing on one or more external network services. The external function system can comprise a particular external function for a particular external service, an outbound serializer function, and an inbound serializer function that are linked with the particular external function. The outbound serializer function can be configured to transform the data of a query from a database format to a different format of the particular external network service. The inbound deserializer function can be configured to receive data returned from the external service and transform the data to the format of the database.

Abstract: External functions can provide users of a database system the ability to invoke external services while operating from inside the database system. The techniques described herein address may detect throttling by the external system and automatically adjust or modify a request rate accordingly. Also, the techniques described herein may increase the request rate to optimize the use of the resources provided by the external system. These techniques allow the database system to improve efficiency when working with external services, which may or may not be elastic or scalable. Moreover, these techniques can decrease query execution times and reduce costs.

Abstract: A query referencing an external table function provided by a remote software component is received. Requests to execute the external table function on input data are sent to a proxy service. A first request includes a batch of input rows from the input data. A first response to the first request received from the proxy service includes a first portion of result data and a pagination token. The pagination token indicates that at least a second portion of the result data corresponding to the first batch of input rows is to be obtained from the remote software component. Based on the pagination token, a second request is sent to obtain the second portion of the result data. One or more responses are received from the proxy service that comprise at least the second portion of the result data. The result data is processed according to the query.

Abstract: A query referencing an external table function provided by a remote software component is received. Requests to execute the external table function on input data are sent to a proxy service. A first request includes a batch of input rows from the input data. A first response to the first request received from the proxy service includes a first portion of result data and a pagination token. The pagination token indicates that at least a second portion of the result data corresponding to the first batch of input rows is to be obtained from the remote software component. Based on the pagination token, a second request is sent to obtain the second portion of the result data. One or more responses are received from the proxy service that comprise at least the second portion of the result data. The result data is processed according to the query.