Abstract: Techniques for a distributed data processing application service in a cloud provider network are described. A virtual machine bundle is obtained, the virtual machine bundle including a copy of a memory and one or more registers of a paused virtual machine executing a distributed data processing application runtime. A request to launch a virtual machine to execute a distributed data processing application runtime is received, the request to launch the virtual machine including a bundle identifier that identifies the virtual machine bundle. The virtual machine is launched based at least in part on the virtual machine bundle. The virtual machine sends a message to register the virtual machine with a cluster of virtual machines.

Abstract: In response to receiving a parallel processing job from a customer, a system operated by a computing resource service provider allocates and configures a cluster of computer systems capable of executing the job. In an embodiment, each computer system is configured with a first network stack that allows access to resources of the computing resource service provider and a second network stack that allows access to resources of the customer. In an embodiment, the state of the cluster is monitored by the system via the first network stack. In an embodiment, the system deploys a set of tasks on the cluster for fulfilling the processing job. In an embodiment, the tasks have access to the second network stack so that each task can access resources of the customer.

Abstract: A system operated by a computing resource service provider maintains a pool of computing resources for use in processing job requests submitted by customers. The system allocates computing resources to the pool in anticipation of future resource needs. In various embodiments, future resource needs can be estimated based on scheduled jobs, or historical job information. In an embodiment, the computing resources are virtual computer systems which may be arranged in a cluster. In response to receiving a parallel processing job from a customer, the system reserves computing resources from the pool for performing the job. In an embodiment, the reserved resources are configured with a network namespace that is able to access to a customer's resources.