Abstract: Systems and methods are provided to implement a fast recovery process in a partitioned replicated data store. In some embodiments, the data store is configured to store an object in a plurality of partitions and replicate data in each partition in a group of replica nodes to satisfy a durability model. In response to a replica failure, the data store performs a split operation to create a plurality of new partitions. The partition's data is split into subsets corresponding to the new partitions. The subsets are transmitted, in parallel, from the surviving replica nodes of the partition to new replica nodes in the new partitions. The new partitions then replicate respective subsets of data in their respective replication groups using a chained replication technique. The recovery process allows the data store to return into compliance with the durability model more quickly, by parallelizing the copying of data.

Abstract: A lightweight filesystem may be provided for remote storage caching. A filesystem may maintain a persistent cache for a data set stored as data files in immutable data objects in a remote data store. Filesystem metadata may be evaluated to determine whether a portion of a data file is stored in the persistent cache according to an offset and length specified in a request. If in the persistent cache, data obtained from a data block in the persistent cache may be returned. If not in the persistent cache, then the remote data store may be accessed and the data file in the immutable data object read to obtain the portion of the data file.

Abstract: Time and value ordering may be applied for items stored in data backups. A change log that persists changes to a data set may be updated with changes and used to update an in-memory table for the data set, which describes changes to items up to a current time. An event may be detected to seal the in-memory table from subsequent updates and a persistent data object that orders the items in the in-memory according to both keys of the respective items and the respective time values of the items, as stored in the change log, may be generated and stored as part of a backup for the data set.

Abstract: Time and value ordering may be applied for items stored in data backups. A change log that persists changes to a data set may be updated with changes and used to update an in-memory table for the data set, which describes changes to items up to a current time. An event may be detected to seal the in-memory table from subsequent updates and a persistent data object that orders the items in the in-memory according to both keys of the respective items and the respective time values of the items, as stored in the change log, may be generated and stored as part of a backup for the data set.

Abstract: A distributed, web-services based storage system. A system may include a web services interface configured to receive, according to a web services protocol, a given client request for access to a given data object, the request including a key value corresponding to the object. The system may also include storage nodes configured to store replicas of the objects, where each replica is accessible via a respective unique locator value, and a keymap instance configured to store a respective keymap entry for each object. For the given object, the respective keymap entry includes the key value and each locator value corresponding to replicas of the object. A coordinator may receive the given client request from the web services interface, responsively access the keymap instance to identify locator values corresponding to the key value and, for a particular locator value, retrieve a corresponding replica from a corresponding storage node.

Abstract: A storage gateway serves as an interface between processes on a customer network and a service provider. The storage gateway is located on-premise with the customer processes. To customer processes, it appears that data is stored locally. However, the storage gateway interfaces with a remote storage service to store the data. For cached gateways, the primary data store is a remote data store, while frequently accessed data may be locally cached by the gateway. Reads may be satisfied from the local cache or from virtual data storage; writes are handled so as to appropriately update data blocks in the local cache and/or in virtual data storage. For shadowing gateways, the primary data store is the local data store; reads are passed through to a local data store, and writes are shadowed to virtual data storage as well as being sent to local data store.

Abstract: Updates to projected data subsets may be ordered using conditional operations. When updates to a data set are received, a determination is made according to a schema for a projected subset of the data set as to whether the update is to be propagated to the projected subset of the data set. If the update is to be propagated, a comparison of a version identifier for the update may be made with a version identifier for the subset to determine whether performance of the update is consistent with an ordering of updates performed to the first data set. If the comparison indicates the performance of the update is consistent, then the update may be performed to the projected subset. If not, then the update may not be performed.

Abstract: Offline building of a projected data subset may be performed. A request to create a data set that is a projected subset of data from a source data set may be received. A data store separate from the data store storing the source data set may store a copy of the source data set that is used to replicate items to the projected subset of data according to a schema for the projected data subset. Updates made to the source data set may also be replicated to the projected data subset according to the schema. Conflicts between replicated items and replicated updates to the projected data set may be resolved by comparing a version identifier for the replicated update and replicated item to determine what to store in the projected data subset.

Abstract: A distributed load balancer in which a router receives packets from at least one client and routes packet flows to multiple load balancer (LB) nodes according to a per-flow hashed multipath routing technique. For a given packet flow, the LB nodes randomly select a server node as a target for the packet flow from among multiple server nodes and send a connection request to the server node. A load balancer module on the server node makes the decision on whether to accept or reject the connection based on one or more metrics indicating a respective server's current load. If the module accepts the connection request, a connection is established between the server and the client. Otherwise, the load balancer nodes may select another server node and try again. The connections established between clients and servers pass through the load balancer nodes, but are not terminated at the load balancer nodes.

Abstract: Replicas may be added to a multi-leader replica group for a data set. A new replica may be added to a group of replicas by copying a version of the data set according to a point in time to the new replica. As updates may occur at different ones of the replicas when the new replica is added, updates that occurred at a source replica for the snapshot and other replicas may be replicated by the different replicas according to the point in time at which the snapshot is created without blocking updates at any of the replicas in the replica group.

Abstract: A first run-time environment executing a first instance of an application, and a second run-time environment executing a second instance of the application are established. An indication of an impending commencement of a reduced-capacity phase of operation of the first run-time environment is obtained at a service request receiver. Based at least in part on the indication, the service request receiver directs a service request to the second instance of the application.

Abstract: A computing environment is disclosed that receives from devices requests directed toward services accessible in the environment, and that forwards communications from services in the environment to devices registered with the environment. During a registration process at the environment, devices are assigned a device identifier that is used to identify and authenticate each particular device and requests communicated from and to the device via the environment. The computing environment maintains state information for each device that has been registered with the system. As the device interacts with the system, the state information is updated to reflect the changes in the device. When requests to perform functions are received from devices, the computing environment determines for the particular device and the particular function requested what processing needs to be performed by the environment in response to the request.

Abstract: Resilient consistent hashing techniques are implemented to perform requests among nodes of a distributed system. Different load balancers are assigned different groups of request handling nodes that cache data to perform request. Requests are assigned to different load balancers according to a consistent hashing scheme. Load balancers distribute requests among assigned nodes using a load balancing scheme. Different numbers of nodes can be assigned to different load balancers.

Abstract: Technology for publish-subscribe message updates is provided. In one example, a publish-subscribe messaging method may include receiving a message from a publisher. The message may include a tuple defining message data and a source identifier identifying a data source to update the message. The publication may be queued in a queue for publication to a subscriber. Updated message data may be retrieved from the data source identified in the tuple when the message reaches an update point of the queue. The updated message data may be inserted in the message to generate an updated message which may be published to the subscriber.

Abstract: Methods, apparatus, and computer-accessible storage media for providing redundant storage gateways. A client may create a storage gateway group and add storage gateways to the group. The client may assign one or more volumes on a remote data store to each the storage gateways in the group. Volume data for each storage gateway in the group may be replicated to at least one other storage gateway in the group. If one of the gateways in the group becomes unavailable, one or more other gateways in the group may take over volumes previously assigned to the unavailable gateway, using the replicated data in the group to seamlessly resume gateway operations for the respective volumes. Client processes that previously communicated with the unavailable gateway may be manually or automatically directed to the gateway(s) that are taking over the unavailable gateway's volumes.

Abstract: A transmission format cache may be implemented at a database storage node. Versions of data items stored in a database at the database storage node may be processed according to anticipated access requests to generate an anticipated access responses. The anticipated access responses are then stored in a transmission format cache to provide low latency reads of the data items. The versions of the data items may be processed as a result of updates to the database items. The database storage node may be one of a plurality of database storage nodes implementing a distributed database system with the transmission format cache implementing a portion of a distributed response cache providing low latency, eventually consistent or consistent reads of data items in a distributed database.

Abstract: A distributed, web-services based storage system. A system may include a web services interface configured to receive, according to a web services protocol, a given client request for access to a given data object, the request including a key value corresponding to the object. The system may also include storage nodes configured to store replicas of the objects, where each replica is accessible via a respective unique locator value, and a keymap instance configured to store a respective keymap entry for each object. For the given object, the respective keymap entry includes the key value and each locator value corresponding to replicas of the object. A coordinator may receive the given client request from the web services interface, responsively access the keymap instance to identify locator values corresponding to the key value and, for a particular locator value, retrieve a corresponding replica from a corresponding storage node.

Abstract: A distributed, web-services based storage system. A system may include a web services interface configured to receive, according to a web services protocol, a given client request for access to a given data object, the request including a key value corresponding to the object. The system may also include storage nodes configured to store replicas of the objects, where each replica is accessible via a respective unique locator value, and a keymap instance configured to store a respective keymap entry for each object. For the given object, the respective keymap entry includes the key value and each locator value corresponding to replicas of the object. A coordinator may receive the given client request from the web services interface, responsively access the keymap instance to identify locator values corresponding to the key value and, for a particular locator value, retrieve a corresponding replica from a corresponding storage node.

Abstract: Methods, systems, and computer-readable media for dynamic lease assignments in a time-series database are disclosed. A time-series database determines an assignment of a lease for a tile representing spatial and temporal boundaries of time-series data. The lease is assigned to a first storage node of a plurality of storage nodes. The time-series database routes the elements of the time-series data within the spatial and temporal boundaries to the first storage node based at least in part on the assignment of the lease. The first storage node stores the elements of the time-series data into the tile in a database cluster. Write requests by the first storage node to the tile are validated by the database cluster based at least in part on the assignment of the lease.

Abstract: Disclosed are various embodiments for employing an erasure coding storage scheme and a redundant replication storage scheme in a data storage system. Data objects that are greater than a size threshold and accessed less frequently than an access threshold are stored in an erasure coding scheme, while data objects that are sized less than a size threshold or accessed more often than an access threshold are stored in a redundant replication storage scheme.