Abstract: A database may provide a visualization of restoration operation granularity. Information may be obtained describing possible restoration points within a given period of time for a database. The information may include a respective state of the database associated with a point in time and a respective restoration operation type for the possible restoration points. A graphical representation may be generated illustrating the possible restoration points ordered within the given time period according to the points in time associated with the respective states of the database. The graphical representation may be provided via network-based interface to a client of the database. The graphical representation of the possible restoration points may be updated in response to changes to the given time period or database volume events that modify the possible restoration points.

Abstract: Nodes of a database service may receive a read request to perform a read of a record stored by the database service and a transaction request to perform a transaction to the record. First and second indications of time may be associated with the read and transaction, respectively. A potential read anomaly (e.g., fuzzy read, read skew, etc.) may be detected based, at least in part, on a determination that the first indication of time is within a threshold value of the second indication of time. In response to detecting the potential read anomaly, the read may be performed after the transaction specified by the transaction request, regardless of whether the first indication of time is indicative of an earlier point in time than the second indication of time.

Abstract: Methods and apparatus for reliable distributed messaging are described. A computer system includes a system memory coupled to one or more processors. The system memory comprises at least a non-volatile portion. A particular location within the non-volatile portion is designated as a target location to which a sender module participating in a communication protocol is granted write permission. A receiver module participating in the communication protocol, subsequent to a failure event that results in a loss of data stored in a volatile portion of the system memory, reads a data item written by the sender program at the target location prior to the failure event. The receiver module performs an operation based on contents of the data item.

Abstract: Hybrid quorum policies may be implemented for access requests to state maintained in a distributed system. An update to state maintained in a distributed system may be sent to multiple nodes in the distributed system maintain the state in order to satisfy a hybrid quorum policy for updating the state in the distributed system. The hybrid quorum policy may include individual quorum policies for multiple quorum sets including nodes of the distributed system. Acknowledgments received from nodes in the distributed system indicating that the update is performed may be evaluated according to the individual quorum policies to determine whether the hybrid quorum policy is satisfied. If the hybrid quorum policy is satisfied, then the update to the state may be considered durable.

Abstract: Self-describing data blocks of a minimum atomic write size may be stored for a data store. Data may be received for storage in a data block at a persistent storage device. Metadata may be generated for the data that includes an error detection code which is generated for the data and the metadata together. An individual atomic write operation may write together the data and the metadata in the data block. When accessed, the error detection code is applicable to detect errors and the metadata may also be applicable to determine whether the data is stored for a currently assigned purpose or a previously assigned purpose of the data block.

Abstract: A log-structured data store implementing data backup may implement variable data replication. Write requests may be received at different storage nodes maintaining respective replicas of a portion of a log for data maintained in the log-structured data store. Log records indicating the write requests may be stored in the respective replicas of the log portions at the different storage nodes. The log records may be sent to a backup data store to be durability persisted as part of an archived version of the log. At some of the storage nodes, in response to determining that the log records have been durably persisted in the backup data store, storage space for the log records may be reclaimed. In other remaining storage nodes, the log records may be retained and made accessible for servicing read requests.

Abstract: A database system may maintain a plurality of log records at a distributed storage system. Each of the plurality of log records may be associated with a respective change to a data page. Upon detection of a coalesce event for a particular data page, log records linked to the particular data page may be applied to generate the particular data page in its current state. Detecting the coalesce event may be a determination that the number of log records linked to the particular data page exceeds a threshold.

Abstract: Methods and apparatus for enhancing operating system integrity using non-volatile system memory are described. A computer system includes a system memory coupled to one or more processors. The system memory comprises at least a non-volatile portion. Memory is allocated from the non-volatile portion to store selected metadata associated with an operating system component that supports input-output (I/O) operations. In response to an operation that results in a metadata change at the component, a metadata entry is stored in the non-volatile portion. Subsequent to a failure event, contents of the metadata entry are read from the non-volatile portion to restore a state of the component.

Abstract: A probabilistic data structure is generated for efficient query processing using a histogram for unsorted data in a column of a columnar database. A bucket range size is determined for multiples buckets of a histogram of a column in a columnar database table. In at least some embodiments, the histogram may be a height-balanced histogram. A probabilistic data structure is generated to indicate for which particular buckets in the histogram there is a data value stored in the data block. When an indication of a query directed to the column for select data is received, the probabilistic data structure for each of the data blocks storing data for the column may be examined to determine particular ones of the data blocks which do not need to be read in order to service the query for the select data.

Abstract: Methods and apparatus for enhancing logging using non-volatile system memory are described. A computer system includes a system memory coupled to one or more processors. The system memory comprises at least a non-volatile portion. A range of memory locations within the non-volatile portion is selected as a low-latency high-durability log staging area. A plurality of log records representing respective events detected by one or more executable programs are generated, and at least a subset of the records is stored in the log persistence area. For analysis subsequent to a failure that results in a loss of data stored in a volatile portion of the system memory, log records written to the staging area within a time window immediately prior to the failure event are provided.

Abstract: Nodes of a database service may receive a read request to perform a read of a record stored by the database service and a transaction request to perform a transaction to the record. First and second indications of time may be associated with the read and transaction, respectively. A potential read anomaly (e.g., fuzzy read, read skew, etc.) may be detected based, at least in part, on a determination that the first indication of time is within a threshold value of the second indication of time. In response to detecting the potential read anomaly, the read may be performed after the transaction specified by the transaction request, regardless of whether the first indication of time is indicative of an earlier point in time than the second indication of time.

Abstract: Methods and apparatus for reliable distributed messaging are described. A computer system includes a system memory coupled to one or more processors. The system memory comprises at least a non-volatile portion. A particular location within the non-volatile portion is designated as a target location to which a sender module participating in a communication protocol is granted write permission. A receiver module participating in the communication protocol, subsequent to a failure event that results in a loss of data stored in a volatile portion of the system memory, reads a data item written by the sender program at the target location prior to the failure event. The receiver module performs an operation based on contents of the data item.

Abstract: Data temporarily stored in volatile memory (e.g., RAM) on a host machine can be protected using a component such as an NV-DIMM, which includes components such as an ASIC, non-volatile memory, and a battery. If power is lost to the host, the battery provides the ASIC with the power needed to determine data in the volatile memory that is protected. This protected data then can be transferred to the non-volatile memory on the NV-DIMM. When power is restored, an application or other entity can contact the NV-DIMM to recover the data, which can be transferred over a sideband channel to be restored as appropriate for a prior operation. In at least some embodiments, the NV-DIMM can receive a key over the sideband channel that can be used to encrypt and decrypt the data for further security.

Abstract: A database may provide a visualization of restoration operation granularity. Information may be obtained describing possible restoration points within a given period of time for a database. The information may include a respective state of the database associated with a point in time and a respective restoration operation type for the possible restoration points. A graphical representation may be generated illustrating the possible restoration points ordered within the given time period according to the points in time associated with the respective states of the database. The graphical representation may be provided via network-based interface to a client of the database. The graphical representation of the possible restoration points may be updated in response to changes to the given time period or database volume events that modify the possible restoration points.

Abstract: Hybrid quorum policies may be implemented for access requests to state maintained in a distributed system. An update to state maintained in a distributed system may be sent to multiple nodes in the distributed system maintain the state in order to satisfy a hybrid quorum policy for updating the state in the distributed system. The hybrid quorum policy may include individual quorum policies for multiple quorum sets including nodes of the distributed system. Acknowledgments received from nodes in the distributed system indicating that the update is performed may be evaluated according to the individual quorum policies to determine whether the hybrid quorum policy is satisfied. If the hybrid quorum policy is satisfied, then the update to the state may be considered durable.

Abstract: A distributed database system may perform a single phase commit for transactions involving updates to multiple databases of the distributed database system. A client request may be received that involves updates to multiple database of the distributed database system. The updates may be performed at a front-end database and a back-end database. Log records indicating the updates to the front-end database may be sent to the back-end database. The log records and the updates performed at the back-end database may be committed together as a single phase commit at the back-end database. In the event of a system failure of the front-end database, log records may be requested and received from the back-end database. A restoration of the front-end database may be performed based, at least in part, on the received log records.

Abstract: A storage client and a quorum-based distributed storage system may implement dynamic utilization of bandwidth for a quorum-based distributed storage system. An update at a storage client may be received, and storage nodes of a protection group may be sent a write request indicating the update. In some embodiment, storage nodes that receive the write request may determine whether other storage nodes have not received the update and send the write request to be completed at those other storage nodes. In some embodiments, if a latency threshold is exceeded other storage nodes in the protection group not previously sent the write request may be identified and sent the write request. Based on acknowledgements received from storage nodes in the distributed storage system, it may be determined whether a write quorum requirement is met for a write request.

Abstract: A database system may maintain a plurality of log records at a distributed storage system. Each of the plurality of log records may be associated with a respective change to a data page. Upon detection of a coalesce event for a particular data page, log records linked to the particular data page may be applied to generate the particular data page in its current state. Detecting the coalesce event may be a determination that the number of log records linked to the particular data page exceeds a threshold.

Abstract: A distributed database system may implement log-structured distributed storage using a single log sequence number space. A log for a data volume may be maintained in a log-structured distributed storage system. The log may be segmented across multiple protection groups according to a partitioning of user data for the data volume. Updates to the log may be assigned a log sequence number from a log sequence number space for the data volume. A protection group may be determined for an update according to which partition of user data space the update pertains. Metadata to be included with the log record may indicate a previous log sequence number of a log record maintained at the protection group. The log record may be sent to the protection group and identified as committed based on acknowledgments received from storage nodes implementing the protection group.

Abstract: A network-based services provider may reserve and provision primary resource instance capacity for a given service (e.g., enough compute instances, storage instances, or other virtual resource instances to implement the service) in one or more availability zones, and may designate contingency resource instance capacity for the service in another availability zone (without provisioning or reserving the contingency instances for the exclusive use of the service). For example, the service provider may provision resource instance(s) for a database engine head node in one availability zone and designate resource instance capacity for another database engine head node in another availability zone without instantiating the other database engine head node. While the service operates as expected using the primary resource instance capacity, the contingency resource capacity may be leased to other entities on a spot market. Leases for contingency instance capacity may be revoked when needed for the given service (e.g.