Abstract: A database system provides query processors on demand for accepting customer connections to a database and stores database data in a separate storage layer, via storage nodes each storing a shard or shard replica of the database data. The database system provides a multi-region configuration wherein customers can access a multi-region database from any of multiple regions of a service provider network. In response to a region-wide failure event, query processors are provided on demand in a failover region. Additionally, to ensure sufficient storage node capacity is maintained in a potential failover region, a multi-region control plane distributes load or configuration information to local control planes of each of the regions of the multi-region database to ensure sufficient storage layer scaling is performed to support a failure over event resulting from a region-wide failure.

Abstract: A database system includes a commit layer implemented using a first set of host computing devices and a storage layer implemented using a second set of host computing devices. A control plane of the distributed database system determines a first sharding scheme for the commit layer and a second sharding scheme for the storage layer, wherein the first and second sharding schemes are not required to be the same. Also, in some embodiments, the second sharding scheme used for the storage layer enables overlapping key spaces across the shards of the storage layer, wherein various ones of the shards are optimized for different types of workloads.

Abstract: A database system performs vertical scaling of a storage layer by temporarily increasing a resource allocation of given node and/or shard to allow the node or shard to process a load that exceeds its baseline resource allocation. Additionally, a control plane of the database system performs health checks of the nodes and/or shards of the components of the database system and in response to load conditions exceeding a threshold, performs horizontal scaling of the nodes of the components. The horizontal scaling adds shard replicas or re-shards the nodes to include more shards. The horizontal scaling reduces load on individual nodes and/or shards and alleviates the load conditions that triggered the vertical scaling.

Abstract: A database system may virtualize client connections to query processors to enable the query processors to be used by active connections rather than allowing the query processors to remain idle. Virtualizing the client connections may enable the database system and other systems sharing computing resources with the database system to operate with increased efficiency over a database system which does not virtualize client connections.

Abstract: Compact hosting of database query processors is performed using virtualization snapshots. When a request to access a database is received via a previously established connection from a client for an idle query processor that is not currently executing, a virtualization snapshot may be identified that is associated with the idle query processor. The virtualization snapshot of the query processor may be used to restore the query processor into virtual execution on a host system for the database using an origin snapshot that shares clean pages across multiple query processors on the host system. The request may be performed by the restored query processor in virtual execution on the host system.

Abstract: Synchronous replication for a distributed database system may be performed using an erasure coding scheme. A request that causes a write to a database hosted in a distributed database system is received. A replication message for a synchronous replication technique is generated, then divided and encoded into a number of chunks according to an erasure encoding scheme that allows the replication message to be reassembled with less than the number of chunks. The chunks are sent to another instance of the database which receives and reassembles the replication message from the chunks and responds to acknowledge that the write is committed.

Abstract: Systems and methods are described for service coordination using a PartiQL query. For instance, a method may include selecting a recipe from a plurality of recipes in response to a request from a user device, wherein the recipe includes a plurality of operations to provide a complex function, each of the plurality of operations corresponding to a service of a plurality of services; determining a PartiQL query for an operation of the plurality of operations; executing the PartiQL query to obtain a result; transmitting a call to a service of the plurality of services that corresponds to the operation based on the result; receiving a response from the service; and after all operations of the plurality of operations are called, transmitting a response to the request to the user device.

Abstract: A journal-based database may include items, e.g., organized in a table, associated with a journal. The journal may include a hash-chained set of blocks individually including data representing previous and/or current versions of the individual items. The database may receive a request to redact specific data from the database. The database may search the blocks to determine whether the specific data exists. When the specific data exists in a block of the journal, the database may redate the specific data in the identified block. The database may retain existing metadata of the block such that cryptographic verifiability of the hash-chained blocks may be preserved.