Abstract: Several embodiments using indirection objects to organize data in a cloud storage system are disclosed. An indirection object stores associations to different variants or segments of an object. The indirection object receives requests for the objects and responds with the appropriate variant, segment, or segments of the object. The variant, segment, or segments returned may vary from request to request.

Abstract: In one embodiment, a cloud computing system provides user extensibility by providing a plugin interface for major systems. Plugin interfaces for a compute service, object service, network service, authentication and authorization service, message service, and image service are disclosed. One or more of the plugin interfaces can be used to alter the allocation of virtual to physical resources across multiple services. Compound services and smart allocation facilities are possible with user-directed modification.

Abstract: A system and methods are provided for distributed tracing in a distributed application. In one embodiment, a method includes observing a plurality of messages sent and received among components of the distributed application, generating a probabilistic model of a call flow from observed messages of the distributed system, and constructing a call flow graph based on the probabilistic model for the distributed application. Distributed tracing may include observing messages by performing the subscription-based observation techniques and operations to receive, message traces describing messages being communicated among components of the distributed application. In this regard, the tracing service may merge message traces from different instrumentation points with message traces obtained by observing message queues to generate a probabilistic model and call flow graph.

Abstract: A multi-tenant data center environment includes a dedicated domain having at least one dedicated server associated with a client and a cloud domain having at least one cloud server associated with the client. The cloud server may have a public interface to a public network and a private interface to a private network. In turn, a network device is coupled between the dedicated domain and the public network, and is further coupled to the cloud server via the private network. A controller of the data center may be used to determine presence of the cloud server, and configure the network device to allow certain traffic to pass directly to the dedicated domain, while preventing other traffic from this direct path, based on access controls of the network device.

Abstract: An example method for storing data includes providing a plurality of physical storage pools, each storage pool including a plurality of storage nodes coupled to a network. The method also includes mapping a partition of a plurality of partitions to a set of physical storage pools, where each physical storage pool of the set of physical storage pools is located in a different availability zone, and the storage nodes within an availability zone are subject to a correlated loss of access to stored data. The method further includes receiving a data management request over the network, the data management request being associated with a data object. The method also includes identifying a first partition of the plurality of partitions corresponding to the received data management request and manipulating the data object in the physical storage pools mapped to the first partition in accordance with the data management request.

Abstract: A cloud computing system configured to run virtual machine instances is disclosed. The cloud computing system assigns an identity to each virtual machine instance. When the virtual machine instance accesses initial configuration resources, it provides this identity to the resources to authenticate itself. This allows for flexible and extensible initial configuration of virtual machine instances.

Abstract: A cloud computing system management system including a plurality of computing devices configured to host virtual machine instances, each computing device in the plurality of computing devices including a local agent that continuously evaluates the observed load relative to a utility maximization function. If observed load is higher than a calculated optimal level, individual loading processes are offered for “sale” via a market scheduler. If observed load is lower than a calculated optimal level, then available capacity is offered as a bid via the market scheduler. The market scheduler matches bids with available processes and coordinates the transfer of load from the selling device to the buying device. The offered prices and utility maximization functions can be employed to optimize the performance of the cloud system as a whole.

Abstract: Several different embodiments of a massively scalable object storage system are described. The object storage system is particularly useful for storage in a cloud computing installation whereby shared servers provide resources, software, and data to computers and other devices on demand. In several embodiments, the object storage system includes a ring implementation used to associate object storage commands with particular physical servers such that certain guarantees of consistency, availability, and performance can be met. In other embodiments, the object storage system includes a synchronization protocol used to order operations across a distributed system. In a third set of embodiments, the object storage system includes a metadata management system. In a fourth set of embodiments, the object storage system uses a structured information synchronization system. Features from each set of embodiments can be used to improve the performance and scalability of a cloud computing object storage system.

Abstract: A system for exposing data stored in a cloud computing system to a content delivery network provider includes a database configured to receive and store metadata about the data, the database being implemented in the cloud computing system to store configuration metadata for the data related to the content delivery network, and an origin server configured to receive requests for the data from the content delivery network provider, and configured to provide the data to the content delivery network provider based on the metadata.

Abstract: In one embodiment, there is a method for configuring a virtual machine where there are two storage mechanisms available to the virtual machine: a first storage containing virtual machine operating information, and a second storage including virtual machine configuration information. The configuration information in the second storage is used to configure the virtual machine, including changing the information in the operating storage. The configuration information can pertain to the hypervisor, any logical container within the hypervisor, and any operating environment within on of the logical containers. In a further embodiment, the configuration information from the second storage can be saved and provided to another virtual machine, and used to configure the second virtual machine in a similar fashion. Each virtual machine can have an independent copy of the second storage, or the storage can be mounted in the first machine, unmounted, and then mounted in the second machine.

Abstract: In one embodiment, the present invention can be used to efficiently allocate on-demand resources to a customer of a data center such as a multi-tenant data center having resources dedicated to given customers, as well as on-demand resources that can be flexibly provisioned to customers using a performance zone concept realized via logical switches to present a single logical network to the customer.

Abstract: Several different embodiments of a segmented object storage system are described. The object storage system divides files into a number of object segments, each segment corresponding to a portion of the object, and stores each segment individually in the cloud storage system. The system also generates and stores a manifest file describing the relationship of the various segments to the original data file. Requests to retrieve the segmented file are fulfilled by consulting the manifest file and using the information from the manifest to reconstitute the original data file from the constituent segments. Modifying, appending to, or truncating the object is accomplished by manipulating individual segments and the manifest file. In further embodiments, manipulation of the individual object segments and/or the manifest is used to implement copy-on-write, snapshotting, software transactional memory, and peer-to-peer transmission of the large file.

Abstract: In one embodiment, the present invention is directed to a system with multiple computing hosts each having a hypervisor to provide a virtual environment for the host and one or more containers each including a database instance and at least one database. These databases, and the database instance can be provided as a service to a user of a multi-tenant environment.

Abstract: In one embodiment a method includes receiving a packet including a destination media access control (MAC) address field having a MAC address of a hypervisor and a destination Internet protocol (IP) address field having an IP address of a virtual machine (VM) coupled to the hypervisor. The method further determines a MAC address of the VM using the IP address of the VM and applies the VM MAC address to the destination MAC address field of the packet to forward the packet to the VM.

Abstract: In one embodiment, a distributed database system supporting flexible configuration of data clusters is disclosed. The system includes a cluster manager, an index, and a dataset distributed over one or more database clusters. Where the nodes of the clusters may report ownership of a particular range, the index contains an alternate range. The cluster manager receives requests to access a range of data within database and queries the index to determine the appropriate nodes and/or clusters with which to connect. The cluster manager then directs the requestor to connect to the specified nodes and/or clusters.

Abstract: An example system for indexing heterogeneous resources includes a data module that invokes one or more APIs, and responsive to the invoked one or more APIs, receives a first set of attributes of one or more cloud resources of a first subset of the set of cloud resources and a second set of attributes of one or more cloud resources of a second subset of the set of cloud resources. The system also includes a normalization module that for at least one attribute of the first and second sets of attributes, searches a database for a normalized attribute corresponding to the respective one attribute, normalizes based on the uniform schema the respective attribute, and places the normalized attribute in a data structure. The system further includes an indexing module that indexes the normalized attributes in the data structure.

Abstract: An improved scalable object storage system allows multiple clusters to work together. In one embodiment, a trust and federation relationship is established between a first cluster and a second cluster. This is done by designating a first cluster as a trust root. The trust root receives contact from another cluster, and the two clusters exchange cryptographic credentials. The two clusters mutually authenticate each other based upon the credentials, and optionally relative to a third information service, and establish a service connection. Services from the remote cluster are registered as being available to the cluster designated as the trust root. Multi-cluster gateways can also be designated as the trust root, and joined clusters can be mutually untrusting. Two one-way trust and federation relationships can be set up to form a trusted bidirectional channel.

Abstract: An improved scalable object storage system includes methods and systems allowing multiple clusters to work together. Users working with a first cluster, or with a multi-cluster gateway, can ask for services and have the request or data transparently proxied to a second cluster. This gives transparent cross-cluster replication, as well as multi-cluster compute or storage farms based upon spot availability or various provisioning policies. Vendors providing a cloud storage “frontend” can provide multiple backends simultaneously. In one embodiment, a multi-cluster gateway can have a two, three, or higher-level ring that transparently matches an incoming request with the correct cluster. In the ring, a request is first mapped to an abstract “partition” based on a consistent hash function, and then one or more constrained mappings map the partition number to an actual resource. In another embodiment, the multi-cluster gateway is a dumb gateway, and the rings are located only at the cluster level.

Abstract: A rack management method and system is disclosed. The method includes detecting the presence of a computing device releasably mounted in a frame, the detecting based on an electrical connection established between a configuration bar disposed in a rear portion of the frame and the computing device, and determining a physical location of the computing device within the frame based on the electrical connection. The method also includes retrieving management information about the computing device from a profile storage disposed within the computing device via the electrical connection and storing the management information in a management table, the management table associating the computing device with the physical location within the frame.

Abstract: A rack management method and system is disclosed. The method includes detecting the presence of a computing device releasably mounted in a frame, the detecting based on an electrical connection established between a configuration bar disposed in a rear portion of the frame and the computing device, and determining a physical location of the computing device within the frame based on the electrical connection. The method also includes retrieving management information about the computing device from a profile storage disposed within the computing device via the electrical connection and storing the management information in a management table, the management table associating the computing device with the physical location within the frame.