Abstract: Methods and apparatus for handling failure of traffic forwarding (TF) systems in networks that include multiple zones each including a TF system between a production network and a border network. A TF system advertises routes in its zone and handles egress of packets from sources on the local production network onto the border network. TF systems may also advertise low-priority routes in other zones. If a TF system in a zone fails, sources in the zone may make connection requests to the low-priority routes. Instead of egressing the packets onto the border network, the requests on the low-priority routes are responded to with reset messages. Thus, the sources do not have to wait for a connection timeout, and packets for destinations in the zone are not egressed onto local border networks in other zones and sent through thin pipes between the local border networks.

Abstract: Methods and apparatus for secure data modification using segmented hashing are disclosed. An intermediate device on a data path between a storage service and a client receives a modification request for a data segment of a data chunk of a storage object. The device generates a new chunk hash value for the chunk based on an original chunk hash value of the chunk, an original segment hash value of the segment to be modified, and a new segment hash value computed based on the modification. The device generates a modified version of the chunk based on the modification request, and uploads the modified version and the new chunk hash value to the storage service.

Abstract: Methods and apparatus for uploading data from a sender to a receiver. A data deduplication technique is described that may reduce the bandwidth used in uploading data from the sender to the receiver. In the technique, the receiver, rather than the sender, maintains a fingerprint dictionary for previously uploaded data. When a sender has additional data to be uploaded, the sender extracts fingerprints for units of the data and sends the fingerprints to the receiver. The receiver checks its fingerprint dictionary to determine the data units to be uploaded and notifies the sender of the identified units, which then sends the identified units of data to the receiver. The technique may, for example, be applied in virtualized data store systems to reduce bandwidth usage in uploading data.

Abstract: Methods, apparatus, and computer-accessible storage media for providing a volume-based block storage service and application programming interfaces (APIs) to the service. A block storage service and block storage service APIs may allow processes (applications or appliances) on the service client network to leverage remote, volume-based block storage provided by the service provider. The APIs may provide a standard interface to volume-based block storage operations on a remote data store. The service provider, the service clients, and/or third parties may develop various applications and/or appliances that may, for example, be instantiated in service clients' local networks and that leverage the block storage service via the APIs to create and manage volumes and snapshots on the remote data store and to upload and download data from the volumes and snapshots on the remote data store.

Abstract: Methods and apparatus for secure data modification using segmented hashing are disclosed. An intermediate device on a data path between a storage service and a client receives a modification request for a data segment of a data chunk of a storage object. The device generates a new chunk hash value for the chunk based on an original chunk hash value of the chunk, an original segment hash value of the segment to be modified, and a new segment hash value computed based on the modification. The device generates a modified version of the chunk based on the modification request, and uploads the modified version and the new chunk hash value to the storage service.

Abstract: A distributed system comprises one or more computers implementing a downstream server configured to determine whether it is overloaded and in response, to indicate to one or more upstream servers that the downstream server is in a hotspot situation. The system comprises one or more computers implementing one or more upstream servers configured to respond to receiving the indication of the hotspot situation by shielding the downstream server from subsequent requests, the shielding including serving one or more client requests without requesting service from the downstream server and reporting one or more measures of the shielded requests to the downstream server. The downstream server is further configured to determine whether the hotspot situation still exists, dependent on one or more of the reported measures.

Abstract: Methods, apparatus, and computer-accessible storage media for restoring data from a snapshot to a data volume. The blocks in the volume may be treated as an implicit tree structure, for example a binary tree; each local block corresponds to a block on the snapshot. A local block on the volume may be marked, for example fingerprinted with metadata, to indicate that the local block has not been restored. Initially, the local block at the root node is marked. To restore a local block, the restore process may generate a list indicating all local blocks on a path from the root node of the tree to the target node that have not been restored. The marks in the local blocks are used in generating the list. For each block indicated in the list, children of the block are fingerprinted, and the block is restored from the snapshot.

Abstract: Private network address obfuscation and verification methods and apparatus that may obfuscate private network source addresses embedded in packet header addresses when sending packets from private networks onto or over external, public networks, and that verify incoming packets to the private networks using the obfuscated private network addresses embedded in the incoming packet header destination addresses. Obfuscating the private network addresses embedded in outgoing packets and verifying incoming packets according to the obfuscated content embedded in the destination addresses may help keep the private network addresses of endpoints on the private network hidden in the packet header content on public networks and difficult to detect by entities on the public networks, which may, for example, make malicious activities such as denial of service (DoS) attacks on the private network impractical.

Abstract: Methods, apparatus, and computer-accessible storage media for providing a volume-based block storage service and application programming interfaces (APIs) to the service. A block storage service and block storage service APIs may allow processes (applications or appliances) on the service client network to leverage remote, volume-based block storage provided by the service provider. The APIs may provide a standard interface to volume-based block storage operations on a remote data store. The service provider, the service clients, and/or third parties may develop various applications and/or appliances that may, for example, be instantiated in service clients' local networks and that leverage the block storage service via the APIs to create and manage volumes and snapshots on the remote data store and to upload and download data from the volumes and snapshots on the remote data store.

Abstract: A system for storing data includes a rack, one or more data storage drive assemblies coupled to the rack, and a data control module coupled to the rack. The data storage drive assemblies include one or more drive mechanical modules configured to store data and one or more drive control modules coupled to the drive mechanical modules. The drive control modules control mechanical operations in the drive mechanical modules. The drive mechanical modules and the associated drive control modules are separable from one another without removing the other module from the at least one data storage drive assembly.

Abstract: A packet transmission scheduler that may temporally smooth packet transmission over paths or connections to destinations by scheduling packets for transmission to destinations during transmit windows, and by limiting the amount of data that is scheduled for transmission to each destination in each transmit window. A transmit window limit and state information may be maintained for each destination and used in scheduling packets for the destination in the transmit windows. The scheduler may dynamically adjust the transmit window limits for the destinations according to performance feedback for the connections, allowing the packet transmission scheduler to determine optimal or near-optimal transmit window limits for connections so that packets can be sent to the destinations as quickly as possible at rates that the respective connections can handle without dropping packets or experiencing other problems such as long round trip times.

Abstract: A test system for a distributed load balancer in which a router receives packets from at least one client and routes packet flows to multiple load balancer (LB) nodes, which in turn distribute the packet flows among multiple server nodes. The test system includes message bus technology that enables the distributed load balancer to be run in a single process without requiring load balancing code to be deployed to multiple hosts in a production network. The message bus technology may be implemented in message bus layers of the test system to simulate network segments. The message bus functionality hooks into the IP tables at the kernel level, intercepts packets, and sends the packets up into the message bus process for routing.

Abstract: A distributed load balancer in which a router receives packets from at least one client and routes packet flows to multiple ingress servers. For unknown packet flows, an ingress server cooperates with primary and secondary flow trackers to establish connections to server nodes. For known packet flows, the ingress server sends the packets to target server nodes. The server nodes randomly select egress servers for outgoing packets of the packet flows. The ingress servers, flow trackers, and egress servers are implemented by multiple load balancer nodes in a load balancer node layer. The ingress and egress servers for a given packet flow may be on different load balancer nodes. The load balancer nodes may use a consistent hash function to compute a consistent hash ring for the nodes according to packet flow client/public endpoint pairs so that nodes associated with given packet flows can be located.

Abstract: A computing environment is disclosed that receives from devices requests directed toward services accessible in the environment, and that forwards communications from services in the environment to devices registered with the environment. During a registration process at the environment, devices are assigned a device identifier that is used to identify and authenticate each particular device and requests communicated from and to the device via the environment. The computing environment maintains state information for each device that has been registered with the system. As the device interacts with the system, the state information is updated to reflect the changes in the device. When requests to perform functions are received from devices, the computing environment determines for the particular device and the particular function requested what processing needs to be performed by the environment in response to the request.

Abstract: A computing environment is disclosed that receives from devices requests directed toward services accessible in the environment, and that forwards communications from services in the environment to devices registered with the environment. During a registration process at the environment, devices are assigned a device identifier that is used to identify and authenticate each particular device and requests communicated from and to the device via the environment. The computing environment maintains state information for each device that has been registered with the system. As the device interacts with the system, the state information is updated to reflect the changes in the device. When requests to perform functions are received from devices, the computing environment determines for the particular device and the particular function requested what processing needs to be performed by the environment in response to the request.

Abstract: A computing environment is disclosed that receives from devices requests directed toward services accessible in the environment, and that forwards communications from services in the environment to devices registered with the environment. During a registration process at the environment, devices are assigned a device identifier that is used to identify and authenticate each particular device and requests communicated from and to the device via the environment. The computing environment maintains state information for each device that has been registered with the system. As the device interacts with the system, the state information is updated to reflect the changes in the device. When requests to perform functions are received from devices, the computing environment determines for the particular device and the particular function requested what processing needs to be performed by the environment in response to the request.

Abstract: A system for storing data includes a rack, one or more data storage drive assemblies coupled to the rack, and a data control module coupled to the rack. The data storage drive assemblies include one or more drive mechanical modules configured to store data and one or more drive control modules coupled to the drive mechanical modules. The drive control modules control mechanical operations in the drive mechanical modules. The drive mechanical modules and the associated drive control modules are separable from one another without removing the other module from the at least one data storage drive assembly.

Abstract: A distributed load balancer in which a router receives packets from at least one client and routes packet flows to multiple load balancer (LB) nodes, which in turn distribute the packet flows among multiple server nodes. Each LB node may serve in ingress, egress, and/or flow tracker roles. Each LB node may include a first network interface controller (NIC) that faces the router and a second NIC that faces the server nodes. Each LB node may implement a core packet processing architecture in which packets received at the NICs are distributed among non-blocking input queues of a set of worker cores by receiver cores for the NICs, processed from the input queues by the worker cores, and placed on non-blocking input queues of transmit cores for the NICs.

Abstract: Methods, apparatus, and computer-accessible storage media for shadowing data stored on a local store to a remote store provided by a service provider. A gateway may be configured as a shadowing gateway on a customer network in response to receiving configuration information. The shadowing gateway may receive reads and writes to the local store. The gateway passes the requests to the local store, and also uploads write data indicated by the writes to the service provider to update a snapshot of the local store maintained by the service provider on the remote store. The write data may be buffered to a write log for uploading, and may be uploaded as blocks according to a block storage format used by the service provider. The shadowing process may be transparent to processes on the customer network. The shadowed data may be used to recover data on the local store.

Abstract: A request to retrieve a persistently stored data object is received, the request including a data object identifier that encodes at least storage location information and validation information related to the data object. The data object is retrieved using at least the storage location information to form a retrieved data object, and validation is performed using at least the validation information.