Abstract: Techniques for implementing a single-addressable virtual topology element (VTE) in a virtual topology. A VTE in a virtual topology may be distributed as multiple instantiated elements in a physical topology. However, the multiple instantiated elements are addressable as a single entity. Obtaining information associated with the VTE includes obtaining and aggregating information from each of the instantiated elements. Applying an overall configuration to the VTE includes determining a respective configuration for each instantiated element based on the overall configuration, and applying the respective configuration to each instantiated element.

Abstract: Techniques for implementing a virtual tap in a virtual topology are described. A request to tap a particular link between a set of virtual topology entities (VTEs) is received. The particular link between the set of VTEs is mapped to a link between a set of instantiated entities in a physical topology. One of the instantiated elements implements a tapping function. Additionally or alternatively, a tapping element is instantiated and the tapping element implements the tapping function. The tapping element is not one of the set of instantiated entities. The tapping function includes: applying a tap filter to a set of data; replicating the data that passes the tap filter; and transmitting the replicated data to a virtual endpoint. The replicated data may be referred to as “tapped data.” A user interface engine presents information regarding the particular link between the set of VTEs based on the tapped data.

Abstract: Systems and method for the management of migrations of virtual machine instances are provided. A migration manager monitors the resource usage of a virtual machine instance over time in order to create a migration profile. When migration of a virtual machine instance is desired, the migration manager schedules the migration to occur such that the migration conforms to the migration profile.

Abstract: A virtual tape library system is used to back up data from a client archive system expecting physical tape operations onto logical data containers and/or a metadata store of a storage service by emulating the physical tape operations. For example, a virtual tape library appliance is installed at a customer premise location to interface with a client archive system. The virtual tape library appliance provides virtual interfaces to appear as physical tape library subsystems, such as tape drives and media changing interfaces. However, these virtual interfaces are supported by logical data containers in a storage service and a metadata store. The virtual tape library system allows the client archive system to make requests to import new virtual tapes, export virtual tapes for archiving, store virtual tapes at a virtual location, load and eject virtual tapes into a virtual tape drive and operate on virtual tapes in a virtual tape drive.

Abstract: High-speed processing of packets to, and from, a virtualization environment can be provided while utilizing hardware-based segmentation offload and other such functionality. A hardware vendor such as a network interface card (NIC) manufacturer can enable the hardware to support open and proprietary stateless tunneling in conjunction with a protocol such as single root I/O virtualization (SR-IOV) in order to implement a virtualized overlay network. The hardware can utilize various rules, for example, that can be used by the NIC to perform certain actions, such as to encapsulate egress packets and decapsulate packets.

Abstract: Methods and apparatus for supporting cached volumes at storage gateways are disclosed. A storage gateway appliance is configured to cache at least a portion of a storage object of a remote storage service at local storage devices. In response to a client's write request, directed to at least a portion of a data chunk of the storage object, the appliance stores a data modification indicated in the write request at a storage device, and asynchronously uploads the modification to the storage service. In response to a client's read request, directed to a different portion of the data chunk, the appliance downloads the requested data from the storage service to the storage device, and provides the requested data to the client.

Abstract: Technologies are described herein for migrating network addresses between host computers in a hybrid network environment. Before migrating a network address between a source host computer in a first network and a target host computer in a second network, potentially affected host computers in the networks are notified of the upcoming migration. This may cause an address translation module of an affected host computer to enter a fast polling mode. The networking infrastructures of the first and second networks are then reconfigured to effect transfer of the network address. This may include updating overlay IP mappings for the network address being migrated to point to the target host computer. Finally, the affected host computers are notified that migration of the network address is complete, causing the address translation module of the affected host computer to stop fast polling and return to normal operation with update mappings.

Abstract: A method and apparatus for staged execution pipelining and allocating resource to staged execution pipelines are provided. One or more execution pipelines are established, where each of the one or more execution pipelines includes one or more execution stages. Data is provided to the one or more execution pipelines for processing and resources are allocated to the execution pipeline.

Abstract: Processes and systems are disclosed for selecting a producer system from a number of producer systems to lease to a consumer system. A leasing agent, in response to a request from the consumer system for access to a service at a producer system, can identify a producer system to lease to the lease requestor based, at least in part, on a selection weight associated with each producer system that the leasing agent is assigned. The selection weights can be modified based on status information associated with each of the producer systems. This status information may be obtain from the producer systems and/or from a consumer system that has previously accessed the producer system. The consumer system may provide the status information to the leasing agent as part of the consumer system's lease request.

Abstract: High-speed processing of packets to, and from, a virtualization environment can be provided while utilizing hardware-based segmentation offload and other such functionality. A hardware vendor such as a network interface card (NIC) manufacturer can enable the hardware to support open and proprietary stateless tunneling in conjunction with a protocol such as single root I/O virtualization (SR-IOV) in order to implement a virtualized overlay network. The hardware can utilize various rules, for example, that can be used by the NIC to perform certain actions, such as to encapsulate egress packets and decapsulate packets.

Abstract: Systems and methods are disclosed which facilitate management of thin provisioned data storage. Specifically, portions of thinly provisioned data stores may be deallocated when they contain invalid data, such as data deleted by a user. A user may transmit notifications, which may include delete notifications, such as TRIM commands, to a provider of the data store (or to the data store itself) that data has been deleted. A management component may modify the data store, or metadata corresponding to the data store, to reflect the deletion. The management component may further monitor portions of the data store to determine whether individual portions contain entirely invalid data. If so, the portion may be deallocated from the thin provisioned data store, resulting in more efficient thin provisioning. Deallocation may be enabled even where deletion notifications from a user do not correspond directly to allocated storage portions.

Abstract: Processes and systems are disclosed for selecting a producer system from a number of producer systems to lease to a consumer system. A leasing agent, in response to a request from the consumer system for access to a service at a producer system, can identify a producer system to lease to the lease requestor based, at least in part, on a selection weight associated with each producer system that the leasing agent is assigned. The selection weights can be modified based on status information associated with each of the producer systems. This status information may be obtain from the producer systems and/or from a consumer system that has previously accessed the producer system. The consumer system may provide the status information to the leasing agent as part of the consumer system's lease request.

Abstract: Processes and systems are disclosed for leasing a producer virtual machine on behalf of a consumer virtual machine in an overlay network. The consumer host of the consumer virtual machine can communicate with a set of leasing agents to obtain the identity of a number of producer virtual machines capable of providing the consumer virtual machine with access to a service. When the consumer virtual machine attempts to communicate with a producer system, the consumer host can identify a producer host that hosts a target producer virtual machine and redirect a service request to the producer host.

Abstract: In response to receiving a write request directed to a particular logical block of a storage object, a page of free space (sufficient to accommodate the payload of the write request, but smaller in size than the logical block) of a particular extent that has been selected to store contents of the logical block is allocated. The current size of the extent is smaller than the combined sizes of logical blocks that are mapped to the extent. The page is modified in accordance with a payload indicated in the write request. In response to a subsequent write request directed to the particular extent, a determination is made that the particular extent would violate a free space threshold criterion if the payload of the write request were accommodated, and an extent expansion operation is initiated.

Abstract: A write request directed to a storage object is received at a distributed file storage service. Based on a variable stripe size selection policy, a size of a particular stripe of storage space to be allocated for the storage object is determined, which differs from the size of another stripe allocated earlier for the same storage object. Allocation of storage for the particular stripe at a particular storage device is requested, and if the allocation succeeds, the contents of the storage device are modified in accordance with the write request.

Abstract: Systems and method for the management of migrations of virtual machine instances are provided. A migration manager monitors the resource usage of a virtual machine instance over time in order to create a migration profile. When migration of a virtual machine instance is desired, the migration manager schedules the migration to occur such that the migration conforms to the migration profile.

Abstract: Secure networking processes, such as packet encapsulation and decapsulation, can be executed upstream of a user or guest operating system provisioned on a host machine, where the user has substantially full access to that machine. The processing can be performed on a device such as a network interface card (NIC), which can have a separate network port for communicating with mapping systems or other devices across a cloud or secure network. A virtual image of the NIC can be provided to the user such that the user can still utilize at least some of the NIC functionality. In some embodiments, the NIC can work with a standalone processor or control host in order to offload much of the processing to the control host. The NIC can further handle headers and payload separately where possible, in order to improve the efficiency of processing the various packets.

Abstract: Systems and method for the management of migrations of virtual machine instances are provided. A migration manager monitors a resource usable for migration of a virtual machine instance in order to predict availability of the migration resource. When migration of a virtual machine instance is desired, the migration manager schedules the migration to occur at a future point in time identified based on the predicted availability of the migration resource.

Abstract: Methods and apparatus for supporting cached volumes at storage gateways are disclosed. A storage gateway appliance is configured to cache at least a portion of a storage object of a remote storage service at local storage devices. In response to a client's write request, directed to at least a portion of a data chunk of the storage object, the appliance stores a data modification indicated in the write request at a storage device, and asynchronously uploads the modification to the storage service. In response to a client's read request, directed to a different portion of the data chunk, the appliance downloads the requested data from the storage service to the storage device, and provides the requested data to the client.

Abstract: The state of firmware for devices on a provisioned host machine can be validated independent of the host CPU(s) or other components exposed to the user. A port that is not fully exposed or accessible to the user can be used to perform a validation process on firmware without accessing a CPU of the host device. The firmware can be scanned and a hashing or similar algorithm can be used to determine validation information, such as hash values, for the firmware, which can be compared to validation information stored in a secure location. If the current and stored validation information do not match, one or more remedial actions can be taken to address the firmware being in an unknown or unintended state.