Abstract: Techniques for query execution against relational databases using connection pooling are described. According to some embodiments, a query processing service is disclosed that enables users to execute queries against target relational databases implemented by a relational database service. The service receives query requests originated by a client application at a web service endpoint and identifies a connection to a particular target database. In some examples, the query request is a Hyper Text Transfer Protocol (HTTP) message comprising a query to be executed by a target database instance within the provider network. The service transmits the query for execution at the target database via the connection and obtains a query result based on the execution of the query. The service transforms the query result into a format suitable for the client and transmits a query response to the client.

Abstract: Disclosed herein are embodiments of systems and methods for synchronizing file-catalog table with a file stage. In an embodiment, a data platform receives a notification of a modification to one or more files in a file stage. The file stage includes data storage having a storage location. The data platform updates, based on receiving the notification of the modification, a first file-catalog table for the file stage based on the modification. The first file-catalog table includes a row for each file in the file stage, as well as a column for each of one or more metadata properties of the one or more files in the file stage.

Abstract: In one aspect, a method includes forming, at a database platform, a file stage in a user account on the database platform, the user account being associated with a client, forming, at the database platform, a hidden stage directory table and a data pipeline for the file stage, and binding the data pipeline to the file stage. The method also includes receiving a notification of an event of the file stage. The method also includes identifying the data pipeline corresponding to the file stage. The method also includes updating the hidden stage directory table of the identified data pipeline.

Abstract: The distributed database can implement unstructured data replication using an internal or external storage location. Metadata, such as a directory table that lists the unstructured files, can be replicated across different deployments, followed by replication of the staged data. Replicating the staged data can be implemented by replication of only the stage metadata or replication of the database files between the deployments.

Abstract: Disclosed herein are embodiments of systems and methods for synchronizing file-catalog table with a file stage. In an embodiment, a data platform receives a notification of a modification to one or more files in a file stage. The file stage includes data storage having a storage location. The data platform updates, based on receiving the notification of the modification, a first file-catalog table for the file stage based on the modification. The first file-catalog table includes a row for each file in the file stage, as well as a column for each of one or more metadata properties of the one or more files in the file stage.

Abstract: In one aspect, a method includes forming, at a database platform, a file stage in a user account on the database platform, the user account being associated with a client, forming, at the database platform, a hidden stage directory table and a data pipeline for the file stage, and binding the data pipeline to the file stage. The method also includes receiving a notification of an event of the file stage. The method also includes identifying the data pipeline corresponding to the file stage. The method also includes updating the hidden stage directory table of the identified data pipeline.

Abstract: In response to receiving a parallel processing job from a customer, a system operated by a computing resource service provider allocates and configures a cluster of computer systems capable of executing the job. In an embodiment, each computer system is configured with a first network stack that allows access to resources of the computing resource service provider and a second network stack that allows access to resources of the customer. In an embodiment, the state of the cluster is monitored by the system via the first network stack. In an embodiment, the system deploys a set of tasks on the cluster for fulfilling the processing job. In an embodiment, the tasks have access to the second network stack so that each task can access resources of the customer.

Abstract: Disclosed herein are embodiments of systems and methods for synchronizing file-catalog table with a file stage. In an embodiment, a data platform receives a notification of a modification to one or more files in a file stage. The file stage includes data storage having a storage location. The data platform updates, based on receiving the notification of the modification, a first file-catalog table for the file stage based on the modification. The first file-catalog table includes a row for each file in the file stage, as well as a column for each of one or more metadata properties of the one or more files in the file stage.

Abstract: Disclosed herein are systems and methods for population of a file-catalog table for a file stage in a user account on a data platform. In an embodiment, a data platform receives, from a client associated with a user account, a request to populate a file-catalog table of the user account based on a plurality of files stored in a file stage of the user account. The data platform responsively executes a list-files table function with respect to the file stage to generate a database-table object having a row for each file stored in the file stage. The data platform populates the file-catalog table of the user account based on the database-table object generated by the list-files table function.

Abstract: Disclosed herein are systems and methods for population of a file-catalog table for a file stage in a user account on a data platform. In an embodiment, a data platform receives, from a client associated with a user account, a request to populate a file-catalog table of the user account based on a plurality of files stored in a file stage of the user account. The data platform responsively executes a list-files table function with respect to the file stage to generate a database-table object having a row for each file stored in the file stage. The data platform populates the file-catalog table of the user account based on the database-table object generated by the list-files table function.

Abstract: Disclosed herein are systems and methods for implementing a file-catalog table for a file stage. In an embodiment, a data platform receives, from a client, a request that the data platform create a file-catalog table for a file stage of a user account associated with the client, where the file stage contains one or more files at a storage location. The user account includes a file-stage object that corresponds to the file stage and that includes an indication of the storage location. In response to receiving the request, the data platform creates the file-catalog table for the file stage. The file-catalog table lists the one or more files in the file stage, and includes at least two different metadata properties of the files listed in the file-catalog table.

Abstract: Disclosed herein are systems and methods for implementing a file-catalog table for a file stage. In an embodiment, a data platform receives, from a client, a request that the data platform create a file-catalog table for a file stage of a user account associated with the client, where the file stage contains one or more files at a storage location. The user account includes a file-stage object that corresponds to the file stage and that includes an indication of the storage location. In response to receiving the request, the data platform creates the file-catalog table for the file stage. The file-catalog table lists the one or more files in the file stage, and includes at least two different metadata properties of the files listed in the file-catalog table.

Abstract: A system operated by a computing resource service provider maintains a pool of computing resources for use in processing job requests submitted by customers. The system allocates computing resources to the pool in anticipation of future resource needs. In various embodiments, future resource needs can be estimated based on scheduled jobs, or historical job information. In an embodiment, the computing resources are virtual computer systems which may be arranged in a cluster. In response to receiving a parallel processing job from a customer, the system reserves computing resources from the pool for performing the job. In an embodiment, the reserved resources are configured with a network namespace that is able to access to a customer's resources.

Abstract: A data platform provides support for unstructured files, semi-structured files, and structured files. In various different embodiments, the data platform provides functions for obtaining relative paths, absolute paths, and presigned URLs for files in internal and external stages; application programming interfaces (APIs) for integrating with external functions; built-in file-catalog tables; integration with cloud-storage platforms; customized views for flexible file-sharing options, stage-encryption options; notification integrations; table functions for generating file-catalog tables; streams on stages; streams on tables; and other features.

Abstract: A logical communication path is provided between a target virtual machine (VM) and a host or application communicating with the VM. The target VM runs on a hypervisor host that has a hypervisor and a proxy agent. The hypervisor manages execution of the VM. A mapping is maintained indicating which VMs execute on which hosts. When the host or application is to send a message or packet to the target VM, the mapping is consulted and the hypervisor host hosting the target VM is identified. The message or packet, which may identify the target VM, is transmitted to the hypervisor host. A proxy agent at the hypervisor host selects a communication channel between the hypervisor and the target VM. The hypervisor then passes the message or packet through the selected channel to the target VM.

Abstract: A logical communication path is provided between a target virtual machine (VM) and a host or application communicating with the VM. The target VM runs on a hypervisor host that has a hypervisor and a proxy agent. The hypervisor manages execution of the VM. A mapping is maintained indicating which VMs execute on which hosts. When the host or application is to send a message or packet to the target VM, the mapping is consulted and the hypervisor host hosting the target VM is identified. The message or packet, which may identify the target VM, is transmitted to the hypervisor host. A proxy agent at the hypervisor host selects a communication channel between the hypervisor and the target VM. The hypervisor then passes the message or packet through the selected channel to the target VM.

Abstract: A system in which a virtual machine manager determines tasks that are to be performed on virtual machines executing on a host computing system. The host computing system further executes an intermediary virtual machine task management module that receives virtual machine tasks from the virtual machine manager. Upon request from the virtual machines, the intermediary module identifies the tasks that are to be performed on the requesting virtual machine to the requesting virtual machine. The virtual machines may perhaps also initiate the performance of such identified tasks. Since the virtual machine itself is initiating contact with the intermediary module, and is not interacting directly with the virtual machine manager, the virtual machine manager need not be in the same sphere of trust as the virtual machine.

Abstract: A logical communication path is provided between a target virtual machine (VM) and a host or application communicating with the VM. The target VM runs on a hypervisor host that has a hypervisor and a proxy agent. The hypervisor manages execution of the VM. A mapping is maintained indicating which VMs execute on which hosts. When the host or application is to send a message or packet to the target VM, the mapping is consulted and the hypervisor host hosting the target VM is identified. The message or packet, which may identify the target VM, is transmitted to the hypervisor host. A proxy agent at the hypervisor host selects a communication channel between the hypervisor and the target VM. The hypervisor then passes the message or packet through the selected channel to the target VM.

Abstract: A network performance monitor for a computer. The performance monitor maintains a baseline for communication paths between a networked computer and other networked computers connected. The network performance baseline is established and maintained by measuring and updating network performance parameters of the communication paths. The network performance parameters are updated using values obtained during time intervals when the communication path consumed a bandwidth approximately equal to its estimated bandwidth. If congestion is detected on the network or the network is underutilized, values of the network performance parameters obtained during that interval are not used to update the network performance parameters. Some performance parameters in the baseline, such as minimum and maximum bandwidth values may be updated by computing a moving average of each parameter. Asymmetric weight values for updating the moving average may be selected based on a direction of change of a value of a network parameter.

Abstract: A system in which a virtual machine manager determines tasks that are to be performed on virtual machines executing on a host computing system. The host computing system further executes an intermediary virtual machine task management module that receives virtual machine tasks from the virtual machine manager. Upon request from the virtual machines, the intermediary module identifies the tasks that are to be performed on the requesting virtual machine to the requesting virtual machine. The virtual machines may perhaps also initiate the performance of such identified tasks. Since the virtual machine itself is initiating contact with the intermediary module, and is not interacting directly with the virtual machine manager, the virtual machine manager need not be in the same sphere of trust as the virtual machine.