Abstract: A database management system receives a request to process a database query on behalf of a security principal. The database management system determines that processing the database query requires access to an encrypted portion of a file containing data subject to access conditions. The database management system determines that the security principle is authorized to use a key that corresponds to the encrypted portion of the file. The database management system then completes processing of the query by using the key to access the encrypted portion of the file.

Abstract: A multi-framework blockchain service may be implemented with a common interface to manage different types of blockchain networks. Requests to create a blockchain network may be received via an interface for the control plane that triggers the creation of the blockchain network according to an identified workflow. Various operations to change the blockchain network, including membership changes, node additions, governance changes, analytics changes, and monitoring changes may be allowed or denied by the control plane according to a distributed governance policy in effect for the blockchain network.

Abstract: Techniques for storage optimization in a distributed object store are described. A storage optimization service of a provider network monitors changes to data objects in a distributed object store that are part of a data lake and are referenced by a table index. The storage optimization service determines whether particular storage optimizations involving the data objects would be beneficial, prioritizes the ordering of these optimizations with a focus on performing impactful optimizations first, while intelligently scheduling the optimizations to avoid overutilization of available resources.

Abstract: Techniques for a distributed data processing application service in a cloud provider network are described. A virtual machine bundle is obtained, the virtual machine bundle including a copy of a memory and one or more registers of a paused virtual machine executing a distributed data processing application runtime. A request to launch a virtual machine to execute a distributed data processing application runtime is received, the request to launch the virtual machine including a bundle identifier that identifies the virtual machine bundle. The virtual machine is launched based at least in part on the virtual machine bundle. The virtual machine sends a message to register the virtual machine with a cluster of virtual machines.

Abstract: Techniques for indexing large scale datasets are described. A method for indexing large scale datasets can include receiving, by an indexing service, a request to generate an index for a dataset stored in an data storage service, the request including indexing information for the dataset, determining, by the indexing service, an index type based at least on the dataset, generating, by the indexing service, the index based at least on the indexing information and the index type, and receiving, by the indexing service, a request from a query service to identify a subset of the dataset using the index.

Abstract: Techniques are described for an interactive query service that enables users to query data stored at a federated collection of data sources. An interactive query service provides interfaces that enable users to configure the interactive query service to query any number of heterogeneous data sources pertinent to a user. In general, the configuration of a data source can include identification of: a data source type, access configurations related to accessing the data source, and in some cases metadata describing a structure of the data stored by the data source (for example, a data catalog describing schemas, tables, columns, partitions, datatypes, or other metadata associated with the stored data). Once configured, an interactive query service can receive and execute queries that involve data stored at any combination of a user's data sources, where the queries may be expressed using a standard query language such as the Structured Query Language (SQL).

Abstract: Access across private networks may be enabled for blockchain networks in a managed blockchain service. Requests to enable access for a node hosted in a private network to one or more nodes hosted in a different private network that hosts the blockchain network as part of a managed blockchain service may be received. Routes to enable the access may be determined and added to networking devices so that communications between the node in the private network to specified nodes in the private network of the managed blockchain service may be correctly routed.

Abstract: Techniques for cooperative preemption of resources in a distributed multi-tenant resource pool are described. A query execution service manages the allocation of computing resources in a provider network by identifying computing nodes in the provider network to preempt. The service causes the computing nodes to be removed from further executing tasks as part of processing their queries in a first group of nodes in the provider network while other computing nodes in the first group continue to execute tasks as part of processing the queries. The service then adds the first computing node into a second group of nodes in the provider network to execute tasks as part of processing other queries in the provider network.

Abstract: A shared resource service allows multiple clients to agree on rules for accessing a shared resource (e.g., a shared database or a shared service). The shared resource service also allows the clients to make changes to the rules (e.g., via consensus or majority vote). The clients use the shared resource service to enforce the rules, without having to trust each other to follow the rules when accessing the shared resource. First, the clients agree on a set of initial rules for accessing the shared resource and a set of initial rules for making changes to the rules. After the rules are initialized, then the clients can begin using the shared resource in accordance with the agreed upon rules. In response to a request for accessing the shared resource or a request for changing rules, the shared resource service enforce the applicable rules.

Abstract: Techniques are described for an interactive query service that enables users to query data stored at a federated collection of data sources. An interactive query service provides interfaces that enable users to configure the interactive query service to query any number of heterogeneous data sources pertinent to a user. In general, the configuration of a data source can include identification of: a data source type, access configurations related to accessing the data source, and in some cases metadata describing a structure of the data stored by the data source (for example, a data catalog describing schemas, tables, columns, partitions, datatypes, or other metadata associated with the stored data). Once configured, an interactive query service can receive and execute queries that involve data stored at any combination of a user's data sources, where the queries may be expressed using a standard query language such as the Structured Query Language (SQL).

Abstract: Independent scheduling of resources may be performed for distributed data processing programs. When a request is received that causes the performance of a distributed data processing program, a plan to perform the distributed data processing program may be generated. Different resources may be obtained to perform different portions of the distributed data processing program according to different portions of the generated plan. A result of the request may be provided to a user. In some embodiments, resources obtained for different portions of the distributed data processing program may be returned for use by other programs.

Abstract: Techniques for providing an on-demand development environment are described. A service of a provider network receives a request to launch a development environment, such as a notebook, from an electronic device. The service obtains an identification of a computing cluster hosted by the provider network. The service obtains an identification of a compute instance hosted within the provider network, the compute instance executing a software environment to host one or more development environments. The service causes the compute instance to launch a development environment. The service sends a message to configure the launched development environment to execute a computer program written in the development environment using the computing cluster. The service generates a token to secure communications between the electronic device and the development environment and sends the token to an originator of the request.

Abstract: Partitions of a database table may be predicted for processing database queries. When a database query to a database is received, partitions of the database table that may exist may be predicted according to a partitioning scheme. The predicted partitions may be accessed in order to perform the query. In some embodiments, the partitioning scheme may be received from a user of the database. In some embodiments, the partitioning scheme may be obtained by analyzing the database table.

Abstract: A multi-framework blockchain service may be implemented with a common interface to manage different types of blockchain networks. Requests to create a blockchain network may be received via an interface for the control plane that triggers the creation of the blockchain network according to an identified workflow. Various operations to change the blockchain network, including membership changes, node additions, governance changes, analytics changes, and monitoring changes may be allowed or denied by the control plane according to a distributed governance policy in effect for the blockchain network.

Abstract: Access across private networks may be enabled for blockchain networks in a managed blockchain service. Requests to enable access for a node hosted in a private network to one or more nodes hosted in a different private network that hosts the blockchain network as part of a managed blockchain service may be received. Routes to enable the access may be determined and added to networking devices so that communications between the node in the private network to specified nodes in the private network of the managed blockchain service may be correctly routed.

Abstract: Differences in the way a network and a host operating system handle multicast traffic is used to create a passive monitoring construct that measures both multicast availability and isolation. In one aspect, a computer registers a multicast group with a network to multicast a simple heartbeat message; consumer computers look for this heartbeat to detect multicast failures. In another aspect, a heartbeat is generated on the same multicast group that is being monitored but on a different port than the application traffic for this multicast group. The network routes the heartbeat packets identically to those of the application data but the heartbeat packets are instead delivered to a monitoring process and not the end application on arrival at the consumer computer. A correlating computer serves as a stateful warehouse of all multicast publisher and consumer information using information obtained by client-side agents.

Abstract: A system includes a third party authority in communication with a client computer and a target computer. The third party authority is configured to receive a request including authentication information and an access request from the client computer. The third party authority is configured to authenticate the client computer based on the authentication information and to process the access request to grant the client computer access to the target computer to perform a task on the target computer, the access request including the task. The third party authority is further configured to send an access token to the client computer to access the target computer to perform the task, to receive the access token from the target computer for validation, to validate the received access token based on the request for the target computer to process the task, and to grant the target computer permission to process the task upon validation.