Abstract: Attempts to update confirmation information or firmware for a hardware device can be monitored using a secure counter that is configured to monotonically adjust a current value of the secure counter for each update or update attempt. The value of the counter can be determined every time the validity of the firmware is confirmed, and this value can be stored to a secure location. At subsequent times, such as during a boot process, the actual value of the counter can be determined and compared with the expected value. If the values do not match, such that the firmware may be in an unexpected state, an action can be taken, such as to prevent access to, or isolate, the hardware until such time as the firmware can be validated or updated to an expected state.

Abstract: Virtual machines may migrate between heterogeneous sets of implementation resources in a manner that allows the virtual machines to efficiently and effectively adapt to new implementation resources. Furthermore, virtual machines may change types during migration without terminating the virtual machines. Migration templates may be established to manage migration of sets of virtual machines between sets of implementation resources and/or virtual machine types. Migration templates may be established based at least in part on information provided by migration agents added to the virtual machines under consideration for migration. The migration agents may detect and augment relevant virtual machine capabilities, as well as trigger reconfiguration of virtual machine components in accordance with migration templates.

Abstract: High-speed processing of packets to, and from, a virtualization environment can be provided while utilizing hardware-based segmentation offload and other such functionality. A hardware vendor such as a network interface card (NIC) manufacturer can enable the hardware to support open and proprietary stateless tunneling in conjunction with a protocol such as single root I/O virtualization (SR-IOV) in order to implement a virtualized overlay network. The hardware can utilize various rules, for example, that can be used by the NIC to perform certain actions, such as to encapsulate egress packets and decapsulate packets.

Abstract: High-speed processing of packets to, and from, a virtualization environment can be provided while utilizing segmentation offload and other such functionality of commodity hardware. Virtualization information can be added to extension portions of protocol headers, for example, such that the payload portion is unchanged and, when physical address information is added to a frame, a frame can be processed using commodity hardware. In some embodiments, the virtualization information can be hashed and added to the payload or stream at, or relative to, various segmentation boundaries, such that the virtualization or additional header information will only be added to a subset of the packets once segmented, thereby reducing the necessary overhead. Further, the hashing of the information can allow for reconstruction of the virtualization information upon desegmentation even in the event of packet loss.

Abstract: In a system having multiple processors, idle processors are wakened in anticipation of tasks that may be subsequently queued. When interrupting a first processor to execute a particular task, a scheduler may also send interrupts to idle or otherwise available processors, instructing the idle processors to begin monitoring task queues and to find and execute compatible tasks that may be subsequently queued.

Abstract: A receiver-side deduplication architecture for data storage systems, for example remote data storage systems that use block-based data storage and that provide the data storage to client(s) via a network. The architecture may provide network deduplication by reducing bandwidth usage on communications channel(s) between the client(s) and the data storage systems. The architecture may leverage block storage technology of a data store provided to clients by a data store provider and caching technology to implement a deduplication data dictionary. The deduplication data dictionary includes deduplication data blocks stored in the data store and a mapping tier that leverages caching technology to store and maintain a store of key/value pairs that map data block fingerprints to deduplication data blocks in the data store.

Abstract: Information about the transmission of packets or other information can be inferred based at least in part upon the state of one or more queues used to transmit that information. In a networking example, a hook can be added to a free buffer API call from a queue of a NIC driver. When a packet is transmitted and a buffer freed, the hook can cause information for that packet to be transmitted to an appropriate location, such as a network traffic control component or control plane component, whereby that information can be compared with packet, source, and other such information to infer which packets have been transmitted, which packets are pending, and other such information. This information can be used for various purposes, such as to dynamically adjust the allocation of a resource (e.g., a NIC) to various sources based at least in part upon the monitored behavior.

Abstract: At a particular node of a storage service to which connections have been established on behalf of one or more clients, respective workload indicators are collected from a set of peer nodes of the storage service. A determination is made at the particular node that (a) a local workload metric exceeds a connection rebalancing threshold, and (b) a peer capacity availability criterion has been met. The peer capacity availability criterion may be determined from the respective workload indicators. In response to the determination, a particular client connection is closed.

Abstract: Methods and apparatus for intelligent cache eviction at storage gateways are disclosed. A system comprises computing devices configured to determine whether the number of free chunks of storage at a storage appliance for caching portions of a storage object is below a threshold value. If the number is below the threshold, the computing devices identify an eviction set of chunks to be freed, and generate a respective new instance identifier for each chunk of the eviction set. The identifier of a given chunk may be used to determine a validity of a block of the chunk. The devices store, within metadata storage of the appliance, the new instance identifiers of the eviction set, and indicate that the chunks of the eviction set are available for caching data of the storage object.

Abstract: Information about the transmission of packets or other information can be inferred based at least in part upon the state of one or more queues used to transmit that information. In a networking example, a hook can be added to a free buffer API call from a queue of a NIC driver. When a packet is transmitted and a buffer freed, the hook can cause information for that packet to be transmitted to an appropriate location, such as a network traffic control component or control plane component, whereby that information can be compared with packet, source, and other such information to infer which packets have been transmitted, which packets are pending, and other such information. This information can be used for various purposes, such as to dynamically adjust the allocation of a resource (e.g., a NIC) to various sources based at least in part upon the monitored behavior.

Abstract: Methods and apparatus for efficient recovery of cached volumes at storage gateways are disclosed. To recover, after an unplanned shutdown, a storage gateway appliance configured to cache chunks of a storage object, chunk metadata corresponding to a particular chunk is read into an in-memory metadata region from a first metadata location. Based on analysis of the chunk metadata, a validation requirement indication for the particular chunk is stored, and the chunk is designated as being accessible for client I/O requests. In response to receiving a subsequent I/O request targeted to the particular chunk, the chunk metadata is validated using a different metadata location prior to performing the requested I/O operation.

Abstract: Methods and apparatus for supporting cached volumes at storage gateways are disclosed. A storage gateway appliance is configured to cache at least a portion of a storage object of a remote storage service at local storage devices. In response to a client's write request, directed to at least a portion of a data chunk of the storage object, the appliance stores a data modification indicated in the write request at a storage device, and asynchronously uploads the modification to the storage service. In response to a client's read request, directed to a different portion of the data chunk, the appliance downloads the requested data from the storage service to the storage device, and provides the requested data to the client.

Abstract: Systems and method for the management of migrations of virtual machine instances are provided. A migration manager monitors the resource usage of a virtual machine instance over time in order to create a migration profile. When migration of a virtual machine instance is desired, the migration manager schedules the migration to occur such that the migration conforms to the migration profile.

Abstract: Technologies are described herein for utilizing shared storage for efficient VM-HA. During the replication of the state of a primary virtual machine to a replicated virtual machine hosted on a secondary host, a request from the primary virtual machine is received to write a dirty page of guest memory to a shared storage device. Upon receiving the request, replication metadata indicating the location of the page of guest memory on the shared storage device is sent to the secondary host computer causing the secondary host computer to retrieve the dirty page of guest memory from the shared storage device during synchronization instead of the dirty page of guest memory having to be copied directly from the primary virtual machine to the secondary host computer.

Abstract: Systems and method for the management of migrations of virtual machine instances are provided. A migration manager monitors a resource usable for migration of a virtual machine instance in order to predict availability of the migration resource. When migration of a virtual machine instance is desired, the migration manager schedules the migration to occur at a future point in time identified based on the predicted availability of the migration resource.

Abstract: Technologies are described herein for performing live migration of virtual machines in a hybrid network environment. A network tunnel infrastructure is established between a first network and a second network. A secure communication channel is then created over the network tunnel infrastructure between a source host computer hosting the virtual machine and a target host computer. Live migration of the virtual machine from the source host computer to the target host computer is performed using the secure communication channel, and the network address of the virtual machine is transferred to the target host computer. The operation of the fully migrated virtual machine may then be resumed on the target host computer.

Abstract: High-speed processing of packets to, and from, a virtualization environment can be provided while utilizing hardware-based segmentation offload and other such functionality. A hardware vendor such as a network interface card (NIC) manufacturer can enable the hardware to support open and proprietary stateless tunneling in conjunction with a protocol such as single root I/O virtualization (SR-IOV) in order to implement a virtualized overlay network. The hardware can utilize various rules, for example, that can be used by the NIC to perform certain actions, such as to encapsulate egress packets and decapsulate packets.

Abstract: A snapshot of a volume is taken by proactive uploading of scheduled snapshot data before the scheduled snapshot time has arrived. A volume snapshot schedule of once a day may be set up to a service provider using a speed-limited network connection. Using a determined upload speed of the network connection and a list of changes to the volume since a prior snapshot, a snapshot system may determine an appropriate time to start uploading volume data so that the snapshot may be completed at or after the scheduled snapshot time. By using the list of changes and available bandwidth of the network connection, the snapshot may be completed earlier than had it been started at the time of the snapshot and the available bandwidth of the network connection may be more efficiently used.

Abstract: Technologies are described herein for establishing a secure tunnel infrastructure between host computers in a hybrid network environment. A first network tunnel is established between a border device in a first network and a border device in a second network. A second network tunnel is established between a first host computer in the first network and the border device in the first network. Similarly, a third network tunnel is established between the border device in the second network and a second host computer in the second network. The networking infrastructures of the first and second networks are then configured so that network packets from the first host computer arriving at the border device in the first network through the second network tunnel are sent through the first network tunnel to the border device in the second network, and then through the third network tunnel to the second host computer.

Abstract: A client request, formatted in accordance with a file system interface, is received at an access subsystem of a distributed multi-tenant storage service. After the request is authenticated at the access subsystem, an atomic metadata operation comprising a group of file system metadata modifications is initiated, including a first metadata modification at a first node of a metadata subsystem of the storage service and a second metadata modification at a second node of the metadata subsystem. A plurality of replicas of at least one data modification corresponding to the request are saved at respective storage nodes of the service.