Abstract: A node of a storage service is selected as a coordinator of a distributed transaction involving multiple page-level modifications. The coordinator identifies other nodes as members of a node chain collectively storing physical data pages at which proposed modifications are to be performed, including a decider node responsible for a decision to commit the transaction. The coordinator generates a transaction preparation message comprising a representation of an order of respective commit decisions associated with the proposed modifications, and transmits the message to a selected node of the chain for a sequential propagation along the chain. Each chain node performs a local commit analysis for its changes and stores a record of its intent to commit. If a decision to commit is reached at the decider, the proposed modifications are completed.

Abstract: A write request directed to a storage object is received at a distributed file storage service. Based on a variable stripe size selection policy, a size of a particular stripe of storage space to be allocated for the storage object is determined, which differs from the size of another stripe allocated earlier for the same storage object. Allocation of storage for the particular stripe at a particular storage device is requested, and if the allocation succeeds, the contents of the storage device are modified in accordance with the write request.

Abstract: A directed acyclic graph (DAG) is generated to represent a namespace of a directory. In response to a request to create a new object with a specified name, a hash value bit sequence is computed for the name. A plurality of levels of the DAG are navigated using successive subsequences of the bit sequence to identify a candidate node for storing a new entry corresponding to the specified name. If the candidate node meets a split criterion, the new entry and at least a selected subset of entries of the candidate node's list of entries are distributed among a plurality of DAG nodes, including at least one new DAG node, using respective bit sequences obtained by applying the hash function for each distributed entry.

Abstract: A request for a session identifier for a particular client is transmitted from an access subsystem of a storage service to a metadata subsystem of the service. A session identifier based on a persistent session storage location at which metadata of the client session are stored is received at the access subsystem. The session identifier is cached at the access subsystem prior to its transmission to the client. A lock state indicator generated by the metadata subsystem in response to a particular request from the client during the client session may also be cached at the access subsystem. Subsequent storage requests from the client during the session may be handled by the access subsystem using the cached session identifier and lock state indicator.

Abstract: A set of techniques is described for enabling profile-driven compiler optimization based on cloud-specific information. A service provider may host applications on behalf of multiple users by providing a set of shared resources in a multi-tenant computing environment, where the resources are shared by the various applications hosted thereon. The service provider can collect runtime conditions, resource contention data and other environment-specific information of the shared resources. This gathered information can be provided a profile-driven compiler. The profile-driven compiler can use the information to recompile the source code of the application to produce an optimized version the application that is specifically tuned to run on the shared resources. The running version of the application can then be replaced by the optimized version.

Abstract: Technologies are described herein for establishing a secure tunnel infrastructure between host computers in a hybrid network environment. A first network tunnel is established between a border device in a first network and a border device in a second network. A second network tunnel is established between a first host computer in the first network and the border device in the first network. Similarly, a third network tunnel is established between the border device in the second network and a second host computer in the second network. The networking infrastructures of the first and second networks are then configured so that network packets from the first host computer arriving at the border device in the first network through the second network tunnel are sent through the first network tunnel to the border device in the second network, and then through the third network tunnel to the second host computer.

Abstract: A set of techniques is described for monitoring and analyzing crashes and other malfunctions in a multi-tenant computing environment (e.g. cloud computing environment). The computing environment may host many applications that are executed on different computing resource combinations. The combinations may include varying types and versions of hardware or software resources. A monitoring service is deployed to gather statistical data about the failures occurring in the computing environment. The statistical data is then analyzed to identify abnormally high failure patterns. The failure patterns may be associated with particular computing resource combinations being used to execute particular types of applications. Based on these failure patterns, suggestions can be issued to a user to execute the application using a different computing resource combination.

Abstract: Architectures and techniques for substantially maintaining performance of hyperthreads within processing cores of processors. One technique can include determining that at least one of two or more hyperthreads has entered an idle state. The technique can further include executing an idle workload loop that comprises a set of instructions that substantially simulates execution of the one of the two or more hyperthreads that has entered the idle state.

Abstract: Systems and method for the management of migrations of virtual machine instances are provided. A migration manager monitors the resource usage of a virtual machine instance over time in order to create a migration profile. When migration of a virtual machine instance is desired, the migration manager schedules the migration to occur such that the migration conforms to the migration profile.

Abstract: Attempts to update confirmation information or firmware for a hardware device can be monitored using a secure counter that is configured to monotonically adjust a current value of the secure counter for each update or update attempt. The value of the counter can be determined every time the validity of the firmware is confirmed, and this value can be stored to a secure location. At subsequent times, such as during a boot process, the actual value of the counter can be determined and compared with the expected value. If the values do not match, such that the firmware may be in an unexpected state, an action can be taken, such as to prevent access to, or isolate, the hardware until such time as the firmware can be validated or updated to an expected state.

Abstract: High-speed processing of packets to, and from, a virtualization environment can be provided while utilizing hardware-based segmentation offload and other such functionality. A hardware vendor of an offload device can enable the hardware to support open and proprietary stateless tunneling in conjunction with a protocol such as single root I/O virtualization (SR-IOV) in order to implement a virtualized overlay network. The hardware can utilize various rules, for example, that can be used by the offload device to perform certain actions, such as to encapsulate egress packets and decapsulate packets.

Abstract: High-speed processing of packets to, and from, a virtualization environment can be provided while utilizing segmentation offload and other such functionality of commodity hardware. Virtualization information can be added to extension portions of protocol headers, for example, such that the payload portion is unchanged and, when physical address information is added to a frame, a frame can be processed using commodity hardware. In some embodiments, the virtualization information can be hashed and added to the payload or stream at, or relative to, various segmentation boundaries, such that the virtualization or additional header information will only be added to a subset of the packets once segmented, thereby reducing the necessary overhead. Further, the hashing of the information can allow for reconstruction of the virtualization information upon desegmentation even in the event of packet loss.

Abstract: A set of techniques is described for monitoring and analyzing crashes and other malfunctions in a multi-tenant computing environment (e.g. cloud computing environment). The computing environment may host many applications that are executed on different computing resource combinations. The combinations may include varying types and versions of hardware or software resources. A monitoring service is deployed to gather statistical data about the failures occurring in the computing environment. The statistical data is then analyzed to identify abnormally high failure patterns. The failure patterns may be associated with particular computing resource combinations being used to execute particular types of applications. Based on these failure patterns, suggestions can be issued to a user to execute the application using a different computing resource combination.

Abstract: Disclosed are various embodiments for estimating round-trip times to improve performance of networks. Multiple connections are opened to a network device. Round-trip times associated with sending packets to the network device via the connections are measured. Another connection to the same or a different network device is opened. A round-trip-time estimate for the other connection is initialized based at least in part on the measured round-trip times for the multiple connections, and in some embodiments, network device proximity data.

Abstract: In this disclosure, techniques are described for more efficiently sharing resources across multiple virtual machine instances. For example, techniques are disclosed for allowing additional virtual machine instances to be supported by a single computing system by more efficiently allocating memory to virtual machine instances by providing page swapping in a virtualized environment and/or predictive page swapping. In one embodiment, a virtual memory manager swaps pages predicatively in and/or out of a paging pool based on information from a central processing unit (“CPU”) scheduler. In one embodiment, the CPU scheduler provides scheduling information for virtual machine instances to the virtual memory manager, where the scheduling information allows the virtual memory manager to determine when a virtual machine is scheduled to become active or inactive. The virtual memory manager can then swap-in or swap-out memory pages.

Abstract: Attempts to update confirmation information or firmware for a hardware device can be monitored using a secure counter that is configured to monotonically adjust a current value of the secure counter for each update or update attempt. The value of the counter can be determined every time the validity of the firmware is confirmed, and this value can be stored to a secure location. At subsequent times, such as during a boot process, the actual value of the counter can be determined and compared with the expected value. If the values do not match, such that the firmware may be in an unexpected state, an action can be taken, such as to prevent access to, or isolate, the hardware until such time as the firmware can be validated or updated to an expected state.

Abstract: The state of firmware for devices on a provisioned host machine can be validated independent of the host CPU(s) or other components exposed to the user. A port that is not fully exposed or accessible to the user can be used to perform a validation process on firmware without accessing a CPU of the host device. The firmware can be scanned and a hashing or similar algorithm can be used to determine validation information, such as hash values, for the firmware, which can be compared to validation information stored in a secure location. If the current and stored validation information do not match, one or more remedial actions can be taken to address the firmware being in an unknown or unintended state.

Abstract: Secure networking processes, such as packet encapsulation and decapsulation, can be executed upstream of a user or guest operating system provisioned on a host machine, where the user has substantially full access to that machine. The processing can be performed on a device such as a network interface card (NIC), which can have a separate network port for communicating with mapping systems or other devices across a cloud or secure network. A virtual image of the NIC can be provided to the user such that the user can still utilize at least some of the NIC functionality. In some embodiments, the NIC can work with a standalone processor or control host in order to offload much of the processing to the control host. The NIC can further handle headers and payload separately where possible, in order to improve the efficiency of processing the various packets.

Abstract: High-speed processing of packets to, and from, a virtualization environment can be provided while utilizing segmentation offload and other such functionality of commodity hardware. Virtualization information can be added to extension portions of protocol headers, for example, such that the payload portion is unchanged and, when physical address information is added to a frame, a frame can be processed using commodity hardware. In some embodiments, the virtualization information can be hashed and added to the payload or stream at, or relative to, various segmentation boundaries, such that the virtualization or additional header information will only be added to a subset of the packets once segmented, thereby reducing the necessary overhead. Further, the hashing of the information can allow for reconstruction of the virtualization information upon desegmentation even in the event of packet loss.

Abstract: Various embodiments disclosed herein including systems and methods for improving allocation of computing resources in a virtual machine (VM) environment. Embodiments maintain data relating to how VM image data is stored in storage devices and loaded into volatile memory such as random access memory (RAM). The data is then used to identify common content in the volatile memory that can be shared across VM instances. In some embodiments, multiple VM instances can share at least a portion of a single common VM image loaded into a shared volatile memory.