Abstract: Intelligent query routing may be performed across shards of a scalable database table. A router of a database system may receive an access request directed to one or more database tables. The router may evaluate the access request with respect to metadata obtained for the database tables to determine an assignment distribution of computing resources of the database system to data that can satisfy the access request. The router can select planning locations to perform the access request based on the assignment distribution of the computing resources. The router can cause the access request to be performed according to planning at the selected planning locations.

Abstract: Lease-based consistency may be implemented for databases to handle failovers. A database node may obtain a consistency lease that describes a point in time determined from a time-to-live amount added to a consistent point in time for database data. While the consistency lease is valid, Multi-version Concurrency Control (MVCC) snapshots assigned by the database node can be used to handle requests to access the database data. Once expired, the database node may have to renew the consistency lease in order to continue to handle write and read requests.

Abstract: Optimistic concurrency may be implemented for multiple writers to a database. Writes to a database from different writers may succeed unless a conflict is detected at the data store for the database. For detected conflicts, a request to resolve the conflict between writes may be submitted to a primary writer, which may determine and provide a resolution of the conflict to the conflicting writers. The data store may then be updated to commit the selected write identified by the resolution of the conflict.

Abstract: Systems and methods are disclosed to implement, in a multi-writer database system, a database engine node that optimistically sends queued purge operations to a backend storage system without waiting for acknowledgement of previous operations from the storage system. If a conflict is later detected for a particular operation, the database engine node may retry some portion of the purge operations in the queue. In embodiments, the purge operations are performed as part of a garbage collection phase of a multi-version concurrency control (MVCC) protocol, where unneeded versions of objects previously generated for isolated snapshots are deleted. In embodiments, the operations in the queue may be grouped so that a rejection of one operation in the group will cause the entire group to be retried. Because these purge operations are idempotent and directed to objects that are invisible, they may be freely retried in arbitrary order, without maintaining any rollback state.

Abstract: Methods, systems, and computer-readable media for dynamic prefetching for database queries are disclosed. A query of a database is started according to a first prefetch policy. Before completing the query, the first prefetch policy is changed to a second prefetch policy. A portion of the query is performed according to the second prefetch policy.

Abstract: Updates for transactions to a replication log for a database may be amortized. As updates are received as part of an active transaction, replication log records may be generated. When the size of replication log records exceeds a transaction size threshold, the replication log records may be stored in a remote data store instead of a local data store. When a request to commit the active transaction is received, an replication log for the database may be updated to include the replication log records from a transaction data object in the remote data store.

Abstract: Methods, systems, and computer-readable media for dynamic prefetching for database queries are disclosed. A query of a database is started according to a first prefetch policy. Before completing the query, the first prefetch policy is changed to a second prefetch policy. A portion of the query is performed according to the second prefetch policy.

Abstract: Database commands may be asynchronously forwarded between nodes of a database system. A read-only node may receive a request, such as a read request or write request, and determine that the request should be forwarded to a read-only node for performance. A thread that handles the request may forward the request to the read-only node and register a call-back event for a response to the request and be made available to perform other work. When the response from the read-write node is received, the call-back event may be triggered and return a response to the request using the same or different thread.

Abstract: Techniques for fast online scaling of a database via a split architecture including decoupled storage and compute tiers in a database service are described. A cluster of database (DB) nodes is scaled to add a new DB node. The scaling includes determining a split for data of a first volume managed by an existing DB node. A second DB node is obtained, and the first volume is cloned according to a lightweight copy technique to yield a second volume for use by the second DB node. After the cloning, a set of database modifications are applied to the second volume based on modifications caused by database traffic received by the first DB node, involving the the volume, during the cloning of the first volume. Each DB node may drop the portion of the volume that it does not need according to the split.

Abstract: Storage level query processing may be implemented for processing database queries. Nodes that can access a database may perform parallel processing for at least a portion of a database query. An indication may be received that specifies parallel processing for the database query. The nodes can then be caused to perform the portion of the query as part of providing a result in response to the database query instead of a node, such as a query engine node, that received the database query.

Abstract: A service provider network implements seamless scaling via proxy replay of session state. Upon a trigger, such as a determination to scale a server, a scaled server may be spun up and an identifier of the scaled server provided to a first (existing) server. The first server sends the identification of the second server, and session state information for each of the connections between the first server and the request router, to the request router. For each of the connections, the request router establishes a new connection between the request router and the second (scaled) server, and replays the session state information for the connection to the second server. The request router then routes traffic between each existing client connection (e.g., the same existing client connection which carried traffic delivered to the first server) and the corresponding new connection to the second server.

Abstract: Proxy-based scaling may be performed for databases. A proxy may be implemented for a database that can establish a connection between the proxy and a database engine to perform a database queries received from a client at the proxy. A scaling event may be detected for the database responsive to which the proxy may establish a connection with a new database engine which may, in some embodiments, have different capabilities or resources that address the features or criteria that triggered the scaling event. Session state may be copied from the database engine to the new database engine so that the new database engine may be able to provide access to the database on behalf of requests received from the client through the proxy.

Abstract: Proxy-based scaling may be performed for databases. A proxy may be implemented for a database that can establish a connection between the proxy and a database engine to perform a database queries received from a client at the proxy. A scaling event may be detected for the database responsive to which the proxy may establish a connection with a new database engine which may, in some embodiments, have different capabilities or resources that address the features or criteria that triggered the scaling event. Session state may be copied from the database engine to the new database engine so that the new database engine may be able to provide access to the database on behalf of requests received from the client through the proxy.

Abstract: Automatic consistency is implemented for forwarding writes in a database system. A read-only node may support both read and write requests to a database. The read only node may forward write requests to a read-write node. The read-only node may determine whether to include the forwarded requests in a version of the database for a read request automatically according to a consistency specified for a read request.

Abstract: Systems and methods are disclosed to implement, in a database system, a database engine node that generates write tracking transactions to atomically enforce cross-page constraints for write operations. In embodiments, the database system may include multiple database engine nodes that optimistically perform page writes without checking for potential conflicts at a shared storage system. The storage system is configured to detect and report conflicting writes on the same page. To cause the storage system to report cross-page conflicts for a write, the database engine node creates a transaction that includes the write and a write tracker (WT) operation for a tracked page. The WT operation will cause the storage system to report any conflicting writes to the tracked page before the actual write is accepted by the storage system. If a reported conflict is not favorably resolved, the database engine node will abort the entire transaction, including the actual write.

Abstract: This application relates to multi-level replication filtering for a distributed database. A primary node of the database can generate log records that describe write requests performed to the database. The primary node can, in some embodiments, identify a portion of the data of the database that is stored at a second node, such as data stored by a read replica node that can be used in responding to read requests from clients. The primary node can select log records that are applicable to this identified portion of the data to send to the second node. Log records are not selected based on a determination that those unselected log records are not applicable to the identified portion of the data. The primary node can then send the selected log records to the second node, for the second node to apply the log records to its stored database data.

Abstract: Online restore operations for a database engine can be performed. A request to restore a database to a previous state can be received. Previously stored content of the database, such as snapshot stored prior to the previous state, can be identified along with log records describing changes to be made to the content prior to the previous state. State information in a query engine can be updated based on the previously stored content and log records so that queries can be processed based on the state information at the database restore to the previous state.

Abstract: Online restore operations for a database engine can be performed. A request to restore a database to a previous state can be received. Previously stored content of the database, such as snapshot stored prior to the previous state, can be identified along with log records describing changes to be made to the content prior to the previous state. State information in a query engine can be updated based on the previously stored content and log records so that queries can be processed based on the state information at the database restore to the previous state.

Abstract: Systems and methods are disclosed to implement, in a database system, database engine nodes that implement multi-page writes as an atomic operation. In embodiments, the database system may include multiple database engine nodes that optimistically perform page writes locally without checking for potential conflicts at a shared storage system. The storage system will detect and report conflicting writes on a single page. To ensure that a group of writes to multiple pages occur atomically, the database engine node maintains a transaction context for the group, so that if a page conflict is later detected for any write in the group, the entire group will be rolled back at the database engine node and the storage system. In embodiments, the transaction context will track subsequent writes generated by the node to any page written by the group, so that these subsequent writes are also rolled back with the group.

Abstract: A storage engine may implement dynamic splitting of contentious data pages. Data pages may store data for a table of a data store as part of an indexing structure for the table. Access to the table may be provided by locating the corresponding data pages via the indexing structure. Access contention for different data pages may be monitored. Data pages may be identified for splitting based on the monitoring. A split operation for an identified data page may be formed to store the data on the identified data page on two different data pages so that subsequent access requests for the data are divided between the two data pages. Monitoring of access contention may also be performed to identify data pages for merging in order to consolidate access requests to a single data page.