Abstract: A system that implements a data storage service may store data on behalf of storage service clients. The system may maintain data in multiple replicas that are stored on respective computing nodes in the system. Updates to the stored data and to the membership of replica groups are propagated as replicated log records. A replica receiving a log record may compare metadata in the received log record to corresponding metadata in a log record that was previously appended to its log to determine a response. The metadata may include a sequence number, a lock generation identifier, an epoch identifier, or an indication of an epoch change. The replica may append the received log record to its log, drop the received log record, or cache the received log record for future use. If a log conflict indicates an invalid log stream branch, one or more log records may be deleted.

Abstract: Resource management techniques for shared resources in a distributed system are described. Clients and servers may exchange messages according to an asynchronous messaging protocol that does not guarantee delivery or ordering of messages. A client may send a resource request message including a client timestamp and a measure of client resource demand. The server may allocate a grant of the resource to the client in a manner that prevents resource overload, and indicate the grant to the client via a message including a logical timestamp, the amount of resource granted, the client's original timestamp, and a grant expiration time. The server may acknowledge the grant and cooperatively use the resource in accordance with the grant's terms.

Abstract: A system that implements a scalable data storage service may maintain tables in a non-relational data store on behalf of clients. The system may provide a Web services interface through which service requests are received, and an API usable to request that a table be created, deleted, or described; that an item be stored, retrieved, deleted, or its attributes modified; or that a table be queried (or scanned) with filtered items and/or their attributes returned. An asynchronous workflow may be invoked to create or delete a table. Items stored in tables may be partitioned and indexed using a simple or composite primary key. The system may not impose pre-defined limits on table size, and may employ a flexible schema. The service may provide a best-effort or committed throughput model. The system may automatically scale and/or re-partition tables in response to detecting workload changes, node failures, or other conditions or anomalies.

Abstract: A method and system are disclosed for implementing a distributed lock manager comprising a plurality of nodes that coordinate with one another to maintain a shared state by agreeing to transitions using a distributed consensus protocol and applying those agreed to transitions. A given node of the system periodically propagates a local time from a local clock of the given node to the other nodes. When agreeing to a transition, the system associates the transition with the propagated local clock time. The given node later calculates a maximum staleness of the transition based at least on the current time of the node's local clock and the propagated local time associated with the transition.

Abstract: A system that implements a scalable data storage service may maintain tables in a non-relational data store on behalf of clients. The system may provide a Web services interface through which service requests are received, and an API usable to request that a table be created, deleted, or described; that an item be stored, retrieved, deleted, or its attributes modified; or that a table be queried (or scanned) with filtered items and/or their attributes returned. An asynchronous workflow may be invoked to create or delete a table. Items stored in tables may be partitioned and indexed using a simple or composite primary key. The system may not impose pre-defined limits on table size, and may employ a flexible schema. The service may provide a best-effort or committed throughput model. The system may automatically scale and/or re-partition tables in response to detecting workload changes, node failures, or other conditions or anomalies.

Abstract: A system that implements a scaleable data storage service may maintain tables in a non-relational data store on behalf of service clients. Each table may include multiple items. Each item may include one or more attributes, each containing a name-value pair. The system may provide an API through which clients can query tables maintained by the service. Items may be partitioned and indexed in a table according to a simple or composite primary key contained in all items in the table. A composite primary key may include a hash key attribute, and a range key attribute. The range key attribute may be usable to order items having the same hash key attribute value, and to partition them dependent on a range of range key attribute values. A query request may specify a logical or mathematical expression dependent on range key attribute values and may be directed to multiple partitions.

Abstract: A system and method is disclosed for recording checkpoints in a distributed system. A distributed system comprises one or more computers implementing a plurality of nodes coordinating with one another to maintain a shared state of the distributed system. The system chooses a given one of the plurality of nodes to record a checkpoint of the shared state. In response, the given node records the checkpoint by isolating itself from communication with the other nodes, storing the checkpoint, restarting, and attempting to reinitialize its state from the stored checkpoint. Restarting may include deliberately causing a runtime error in the node. If the reinitialization is successful, the node restores communication with the other nodes and indicates to them that the newly stored checkpoint is valid.

Abstract: A system that implements a scalable data storage service may maintain tables in a non-relational data store on behalf of clients. The system may provide a Web services interface through which service requests are received, and an API usable to request that a table be created, deleted, or described; that an item be stored, retrieved, deleted, or its attributes modified; or that a table be queried (or scanned) with filtered items and/or their attributes returned. An asynchronous workflow may be invoked to create or delete a table. Items stored in tables may be partitioned and indexed using a simple or composite primary key. The system may not impose pre-defined limits on table size, and may employ a flexible schema. The service may provide a best-effort or committed throughput model. The system may automatically scale and/or re-partition tables in response to detecting workload changes, node failures, or other conditions or anomalies.