Abstract: A system that implements a scalable data storage service may maintain tables in a non-relational data store on behalf of clients. The system may provide a Web services interface through which service requests are received, and an API usable to request that a table be created, deleted, or described; that an item be stored, retrieved, deleted, or its attributes modified; or that a table be queried (or scanned) with filtered items and/or their attributes returned. An asynchronous workflow may be invoked to create or delete a table. Items stored in tables may be partitioned and indexed using a simple or composite primary key. The system may not impose pre-defined limits on table size, and may employ a flexible schema. The service may provide a best-effort or committed throughput model. The system may automatically scale and/or re-partition tables in response to detecting workload changes, node failures, or other conditions or anomalies.

Abstract: A system that implements a data storage service may store data on behalf of storage service clients. The system may maintain data in multiple replicas of partitions that are stored on respective computing nodes in the system. The system may split a data partition into two new partitions, and may split the replica group that stored the original partitions into two new replica groups, each storing one of the new partitions. To split the replica group, the master replica may propagate membership changes to the other members of the replica group for adding members to the original replica group and for splitting the expanded replica group into two new replica groups. Subsequent to the split, replicas may attempt to become the master for the original replica group or for a new replica group. If an attempt to become master replica for the original replica group succeeds, the split may fail.

Abstract: A system that implements a scalable data storage service may maintain tables in a non-relational data store on behalf of clients. The system may provide a Web services interface through which service requests are received, and an API usable to request that a table be created, deleted, or described; that an item be stored, retrieved, deleted, or its attributes modified; or that a table be queried (or scanned) with filtered items and/or their attributes returned. An asynchronous workflow may be invoked to create or delete a table. Items stored in tables may be partitioned and indexed using a simple or composite primary key. The system may not impose pre-defined limits on table size, and may employ a flexible schema. The service may provide a best-effort or committed throughput model. The system may automatically scale and/or re-partition tables in response to detecting workload changes, node failures, or other conditions or anomalies.

Abstract: Disclosed are various embodiments for distributing data items within a plurality of nodes. A data item that is subject to a data item update request is updated from a master node to a plurality of slave notes. The update of the data item is determined to be locality-based durable based at least in part on acknowledgements received from the slave nodes. Upon detection that the master node has failed, a new master candidate is determined via an election among the plurality of slave nodes.

Abstract: A system that implements a data storage service may maintain tables in a data store on behalf of clients. The service may maintain table data in multiple replicas of partitions of the data that are stored on respective computing nodes in the system. In response to detecting a failure or fault condition, or receiving a service request from a client to move or copy a partition replica, the data store may copy a partition replica to another computing node using a physical copy mechanism. The physical copy mechanism may copy table data from physical storage locations in which it is stored to physical storage locations allocated to a destination replica on the other computing node. During copying, service requests to modify table data may be logged and applied to the replica being copied. A catch-up operation may be performed to apply modification requests received during copying to the destination replica.

Abstract: A system that implements a scaleable data storage service may maintain tables in a non-relational data store on behalf of service clients. Each table may include multiple items. Each item may include one or more attributes, each containing a name-value pair. The system may provide an API through which clients can query tables maintained by the service. Items may be partitioned and indexed in a table according to a simple or composite primary key contained in all items in the table. A composite primary key may include a hash key attribute, and a range key attribute. The range key attribute may be usable to order items having the same hash key attribute value, and to partition them dependent on a range of range key attribute values. A query request may specify a logical or mathematical expression dependent on range key attribute values and may be directed to multiple partitions.

Abstract: Disclosed are various embodiments for distributing data items within a plurality of nodes. A data item update request is replicated from a master node in a plurality of nodes to a plurality of slave nodes within the plurality of nodes. The replicated data item update request is determined to be locality-based durable. Responsive to the determination that the replicated data item update request is locality-based durable, the data item update request is confirmed to a client, wherein the client had originated the data item update request. Upon failover of the master node to another node within the plurality of nodes, a fault-tolerant failover quorum ensures that all previously confirmed updates are found and recognized by the new master node.

Abstract: A system that implements a data storage service may maintain tables in a data store on behalf of clients. The service may maintain table data in multiple replicas of partitions of the data that are stored on respective computing nodes in the system. In response to detecting a failure or fault condition, or receiving a service request from a client to move or copy a partition replica, the data store may copy a partition replica to another computing node using a physical copy mechanism. The physical copy mechanism may copy table data from physical storage locations in which it is stored to physical storage locations allocated to a destination replica on the other computing node. During copying, service requests to modify table data may be logged and applied to the replica being copied. A catch-up operation may be performed to apply modification requests received during copying to the destination replica.

Abstract: A system that implements a scalable data storage service may maintain tables in a non-relational data store on behalf of clients. The system may provide a Web services interface through which service requests are received, and an API usable to request that a table be created, deleted, or described; that an item be stored, retrieved, deleted, or its attributes modified; or that a table be queried (or scanned) with filtered items and/or their attributes returned. An asynchronous workflow may be invoked to create or delete a table. Items stored in tables may be partitioned and indexed using a simple or composite primary key. The system may not impose pre-defined limits on table size, and may employ a flexible schema. The service may provide a best-effort or committed throughput model. The system may automatically scale and/or re-partition tables in response to detecting workload changes, node failures, or other conditions or anomalies.

Abstract: A system that implements a scalable data storage service may maintain tables in a non-relational data store on behalf of clients. The system may provide a Web services interface through which service requests are received, and an API usable to request that a table be created, deleted, or described; that an item be stored, retrieved, deleted, or its attributes modified; or that a table be queried (or scanned) with filtered items and/or their attributes returned. An asynchronous workflow may be invoked to create or delete a table. Items stored in tables may be partitioned and indexed using a simple or composite primary key. The system may not impose pre-defined limits on table size, and may employ a flexible schema. The service may provide a best-effort or committed throughput model. The system may automatically scale and/or re-partition tables in response to detecting workload changes, node failures, or other conditions or anomalies.

Abstract: A system that implements a scaleable data storage service may maintain tables in a non-relational data store on behalf of service clients. Each table may include multiple items. Each item may include one or more attributes, each containing a name-value pair. The system may provide an API through which clients can query tables maintained by the service. Items may be partitioned and indexed in a table according to a simple or composite primary key contained in all items in the table. A composite primary key may include a hash key attribute, and a range key attribute. The range key attribute may be usable to order items having the same hash key attribute value, and to partition them dependent on a range of range key attribute values. A query request may specify a logical or mathematical expression dependent on range key attribute values and may be directed to multiple partitions.

Abstract: The present invention relates to a system and method of data restoration, for instance, after the occurrence of a user error. In particular, snapshot database can be maintained that stores a copy of database data. The snapshot database does not have to store a complete copy of all data on a source database, but rather shares data that is common to both but not necessarily identical. If an error occurs on the primary database then the database can be reverted back to a point in time prior to the error by replacing source database files with snapshot files. Additionally, an undo component can be employed in conjunction with the snapshot to approach an error to a finer grained point in time. In brief, the present invention can restore a database much faster and simpler, while utilizing less space and resources than conventional data restoration technologies.

Abstract: Disclosed are various embodiments for distributing data items within a plurality of nodes. A data item update request is replicated from a master node in a plurality of nodes to a plurality of slave nodes within the plurality of nodes. The replicated data item update request is determined to be locality-based durable. Responsive to the determination that the replicated data item update request is locality-based durable, the data item update request is confirmed to a client, wherein the client had originated the data item update request. Upon failover of the master node to another node within the plurality of nodes, a fault-tolerant failover quorum ensures that all previously confirmed updates are found and recognized by the new master node.

Abstract: A system that implements a scalable data storage service may maintain tables in a non-relational data store on behalf of clients. The system may provide a Web services interface through which service requests are received, and an API usable to request that a table be created, deleted, or described; that an item be stored, retrieved, deleted, or its attributes modified; or that a table be queried (or scanned) with filtered items and/or their attributes returned. An asynchronous workflow may be invoked to create or delete a table. Items stored in tables may be partitioned and indexed using a simple or composite primary key. The system may not impose pre-defined limits on table size, and may employ a flexible schema. The service may provide a best-effort or committed throughput model. The system may automatically scale and/or re-partition tables in response to detecting workload changes, node failures, or other conditions or anomalies.

Abstract: Disclosed are various embodiments for failover recovery in a distributed data store. A computing device determines whether it is a master data store in a distributed data store. New data item updates are assigned the next transaction identifier in a gapless numbering scheme relative to the most durably persisted entry. New data item updates are also associated with a data store identifier that identifies the new master data store.

Abstract: The present invention relates to a system and method of data restoration, for instance, after the occurrence of a user error. In particular, snapshot database can be maintained that stores a copy of database data. The snapshot database does not have to store a complete copy of all data on a source database, but rather shares data that is common to both but not necessarily identical. If an error occurs on the primary database then the database can be reverted back to a point in time prior to the error by replacing source database files with snapshot files. Additionally, an undo component can be employed in conjunction with the snapshot to approach an error to a finer grained point in time. In brief, the present invention can restore a database much faster and simpler, while utilizing less space and resources than conventional data restoration technologies.

Abstract: The present invention relates to a system and method of data restoration, for instance, after the occurrence of a user error. In particular, snapshot database can be maintained that stores a copy of database data. The snapshot database does not have to store a complete copy of all data on a source database, but rather shares data that is common to both but not necessarily identical. If an error occurs on the primary database then the database can be reverted back to a point in time prior to the error by replacing source database files with snapshot files. Additionally, an undo component can be employed in conjunction with the snapshot to approach an error to a finer grained point in time. In brief, the present invention can restore a database much faster and simpler, while utilizing less space and resources than conventional data restoration technologies.

Abstract: A method of invalidating an unused log file in a database includes examining a discard log information storage area and determining if a discard log exists. If the discard log exists, the log is invalidated. The headers of each of the log files that have been fully invalidated are then invalidated. The discard log is then fully invalidated and a new log file may be generated on top of the discarded log. The method may be interrupted by a failure, a changeover, or crash and will be repeated until it completes.

Abstract: A set of servers takes advantage of the existing data redundancy of a mirrored database to restore page corruptions. The page restore may occur with none of the time and/or administration costs of a restore from backup media and without the data loss associated with repair. Online page restore from a database mirror can be initiated and performed by the computer system automatically upon corruption detection. An entire file or database instead of an individual page or set(s) of pages can be restored. The mechanism can be used both to restore corrupt pages on the principal server from a mirror, or corrupt pages on a mirror from the principal server. Online page restore from a database mirror enables page data recovery without the need to find/load/scan through and apply data and log backups, allowing efficient and potentially automatic data recovery.

Abstract: A method of invalidating an unused log file in a database includes examining a discard log information storage area and determining if a discard log exists. If the discard log exists, the log is invalidated. The headers of each of the log files that have been fully invalidated are then invalidated. The discard log is then fully invalidated and a new log file may be generated on top of the discarded log. The method may be interrupted by a failure, a changeover, or crash and will be repeated until it completes.