Abstract: A system that implements a data storage service may store data on behalf of storage service clients. The system may maintain data in multiple replicas of various partitions that are stored on respective computing nodes in the system. The system may employ a single master failover protocol, usable when a replica attempts to become the master replica for a replica group of which it is a member. Attempting to become the master replica may include acquiring a lock associated with the replica group, and gathering state information from the other replicas in the group. The state information may indicate whether another replica supports the attempt (in which case it is included in a failover quorum) or stores more recent data or metadata than the replica attempting to become the master (in which case synchronization may be required). If the failover quorum includes enough replicas, the replica may become the master.

Abstract: Distributed database management systems may perform range queries over the leading portion of a primary key. Non-random distribution of data may improve performance related to the processing of range queries, but may tend to cause workload to be concentrated on particular partitions. Groups of partitions may be expanded and collapsed based on detection of disproportionate workload. Disproportionate write workload may be distributed among a group of partitions that can subsequently be queried using a federated approach. Disproportionate read workload may be distributed among a group of read-only replicated partitions.

Abstract: A first transaction manager of a partitioned storage group stores a first conditional commit record for a first write of a multi-partition transaction based on a first conflict detection operation. A second transaction manager stores a second conditional commit record for a second write of the transaction based on a second conflict detection operation. A client-side component of the storage group determines that both writes have been conditionally committed, and stores an unconditional commit record in a commit decision repository. A write applier examines the first conditional commit record and the unconditional commit record before propagating the first write to the first partition.

Abstract: A first transaction manager of a partitioned storage group stores a first conditional commit record for a first write of a multi-partition transaction based on a first conflict detection operation. A second transaction manager stores a second conditional commit record for a second write of the transaction based on a second conflict detection operation. A client-side component of the storage group determines that both writes have been conditionally committed, and stores an unconditional commit record in a commit decision repository. A write applier examines the first conditional commit record and the unconditional commit record before propagating the first write to the first partition.

Abstract: At a client-side component of a storage group, a read descriptor generated in response to a read request directed to a first data store is received. The read descriptor includes a state transition indicator corresponding to a write that has been applied at the first data store. A write descriptor indicative of a write that depends on a result of the read request is generated at the client-side component. The read descriptor and the write descriptor are included in a commit request for a candidate transaction at the client-side component, and transmitted to a transaction manager.

Abstract: In response to a read request directed to a first data store of a storage group, a state transition indicator is identified, corresponding to a modification that has been applied at the data store before a response to the read is prepared. A read descriptor that includes the state transition indicator and read repeatability verification metadata is prepared. The metadata can be used to check whether the read request is a repeatable read. The read descriptor is transmitted to a client-side component of the storage group.

Abstract: An automated system may be employed to perform detection, analysis and recovery from faults occurring in a distributed computing system. Faults may be recorded in a metadata store for verification and analysis by an automated fault management process. Diagnostic procedures may confirm detected faults. The automated fault management process may perform recovery workflows involving operations such as rebooting faulting devices and excommunicating unrecoverable computing nodes from affected clusters.

Abstract: A distributed database management system comprising a plurality of computing nodes may distribute data evenly across all nodes. A definition of a primary key that divides the primary key into at least a first key portion and a second key portion may be utilized to locate items related by a first key portion to a specific computing node. Application-consistent queries, local transactions and pivoting operations may be performed on items related by a first key portion.

Abstract: An analytics module may be embedded into an application developed, published, or used by an entity in addition to the owner of the data under analysis. An access token may be submitted by the analytics module to a provider of hosted services. The access token may correspond to an n-dimensional cube containing data at a level of granularity permitted to the application. The access token may incorporate additional policies controlling access to the corresponding n-dimensional cube.

Abstract: An online analytical processing system may comprise an n-dimensional cube partitioned into slices, in which each slice may represent data points at the intersections of fixed and variable dimensions. Computation of data points within a slice may be deferred. A dependency graph may be initially constructed, in which the dependency graph is utilized in a subsequent computation. Calculation of data points may be prioritized based on information indicative of a chance that the data points will be accessed.

Abstract: An analytics module may be embedded into an application developed, published, or used by an entity in addition to the owner of the data under analysis. An access token may be submitted by the analytics module to a provider of hosted services. The access token may correspond to an n-dimensional cube containing data at a level of granularity permitted to the application. The access token may incorporate additional policies controlling access to the corresponding n-dimensional cube.

Abstract: An online analytical processing system may comprise an n-dimensional cube structured using slice-based partitioning in which each slice comprises data points corresponding to a set of dimension values fixed across the slice and a set of dimension values allowed to vary. Slices may be partitioned and replicated across computing nodes. Views of the n-dimensional cube may be partially materialized by determining dependencies between slices. A central data dictionary may maintain information about slices and slice dependencies. Dimensions may be added by adding a new slice without requiring immediate recomputation of existing data points.

Abstract: An online analytical processing system may comprise an n-dimensional cube structured using slice-based partitioning in which each slice comprises one or more hierarchies of data points. A region of a hierarchy may be classified according to computational demands associated with the region. A scaling or replication mechanism may be applied to the region based on the computational demands associated with that region.

Abstract: A system that implements a data storage service may store data on behalf of storage service clients. The system may maintain data in multiple replicas of various partitions that are stored on respective computing nodes in the system. The system may employ a single master failover protocol, usable when a replica attempts to become the master replica for a replica group of which it is a member. Attempting to become the master replica may include acquiring a lock associated with the replica group, and gathering state information from the other replicas in the group. The state information may indicate whether another replica supports the attempt (in which case it is included in a failover quorum) or stores more recent data or metadata than the replica attempting to become the master (in which case synchronization may be required). If the failover quorum includes enough replicas, the replica may become the master.

Abstract: A system that implements a data storage service may store data on behalf of storage service clients. The system may maintain data in multiple replicas of partitions that are stored on respective computing nodes in the system. A master replica for a replica group may increment a membership version indicator for the group, and may propagate metadata (including the membership version indicator) indicating a membership change for the group to other members of the group. Propagating the metadata may include sending a log record containing the metadata to the other replicas to be appended to their respective logs. Once the membership change becomes durable, it may be committed. A replica attempting to become the master of a replica group may determine that another replica in the group has observed a more recent membership version, in which case logs may be synchronized or snipped, or the attempt may be abandoned.

Abstract: A system that implements a data storage service may store data in multiple replicated partitions on respective storage nodes. The selection of the storage nodes (or storage devices thereof) on which to store the partition replicas may be performed by administrative components that are responsible for partition management and resource allocation for respective groups of storage nodes (e.g., based on a global view of resource capacity or usage), or the selection of particular storage devices of a storage node may be determined by the storage node itself (e.g., based on a local view of resource capacity or usage). Placement policies applied at the administrative layer or storage layer may be based on the percentage or amount of provisioned, reserved, or available storage or IOPS capacity on each storage device, and particular placements (or subsequent operations to move partition replicas) may result in an overall resource utilization that is well balanced.

Abstract: A system that implements a scalable data storage service may maintain tables in a data store on behalf of storage service clients. The service may maintain table data in multiple replicas of partitions that are stored on respective computing nodes in the system. In response to detecting an anomaly in the system, detecting a change in data volume on a partition or service request traffic directed to a partition, or receiving a service request from a client to split a partition, the data storage service may create additional copies of a partition replica using a physical copy mechanism. The data storage service may issue a split command defined in an API for the data store to divide the original and additional replicas into multiple replica groups, and to configure each replica group to maintain a respective portion of the table data that was stored in the partition before the split.

Abstract: A system that implements a data storage service may store data on behalf of storage service clients. The system may maintain data in multiple replicas of various partitions that are stored on respective computing nodes in the system. The system may employ a single master failover protocol, usable when a replica attempts to become the master replica for a replica group of which it is a member. Attempting to become the master replica may include acquiring a lock associated with the replica group, and gathering state information from the other replicas in the group. The state information may indicate whether another replica supports the attempt (in which case it is included in a failover quorum) or stores more recent data or metadata than the replica attempting to become the master (in which case synchronization may be required). If the failover quorum includes enough replicas, the replica may become the master.

Abstract: A system that implements a data storage service may store data on behalf of storage service clients. The system may maintain data in multiple replicas of partitions that are stored on respective computing nodes in the system. A master replica for a replica group may increment a membership version indicator for the group, and may propagate metadata (including the membership version indicator) indicating a membership change for the group to other members of the group. Propagating the metadata may include sending a log record containing the metadata to the other replicas to be appended to their respective logs. Once the membership change becomes durable, it may be committed. A replica attempting to become the master of a replica group may determine that another replica in the group has observed a more recent membership version, in which case logs may be synchronized or snipped, or the attempt may be abandoned.

Abstract: A system that implements a data storage service may store data in multiple replicated partitions on respective storage nodes. The selection of the storage nodes (or storage devices thereof) on which to store the partition replicas may be performed by administrative components that are responsible for partition management and resource allocation for respective groups of storage nodes (e.g., based on a global view of resource capacity or usage), or the selection of particular storage devices of a storage node may be determined by the storage node itself (e.g., based on a local view of resource capacity or usage). Placement policies applied at the administrative layer or storage layer may be based on the percentage or amount of provisioned, reserved, or available storage or IOPS capacity on each storage device, and particular placements (or subsequent operations to move partition replicas) may result in an overall resource utilization that is well balanced.