Abstract: Embodiments of a distributed caching system are disclosed that cache data across multiple computing devices on a network. In one embodiment, a first cache system serves as a caching front-end to a distributed cluster of additional cache systems. The first cache system can distribute cache requests to the additional cache systems. The first distributed caching system can also serve as a cache server itself, by storing data on its own internal cache. For example, the first cache system can first attempt to find a requested data item on the internal cache, but, if the lookup results in a cache miss, the first cache system can search the additional cache systems for the data. In some embodiments, the first cache system is configured to multiplex requests to each additional cache system over a single negotiated streaming protocol connection, which allows for network efficiencies and faster detection of failure.

Abstract: Processes and systems are disclosed for partitioning a number of producer systems among a number of leasing agents. A leasing agent can identify a set of available producer systems within an environment and can partition the producer systems among each of the number of leasing agents. The leasing agent can then inform the other leasing agents of the producer systems assigned to each of the leasing agents. As the number of available producer systems change, the partitioning process can be repeated to rebalance the assignment of producer systems among the leasing agents.

Abstract: Processes and systems are disclosed for leasing a consumer system access to a producer system. The consumer system can communicate with a set of leasing agents to obtain the identity of a number of producer systems capable of providing the consumer system with access to a service. Each leasing agent may provide the consumer system with access to a producer system for a period of time. If the consumer system requires further access to a particular producer system, the consumer system can request renewal of the lease from the leasing agent that provided the initial lease of the producer system to the consumer system.

Abstract: Processes are disclosed for embodiments of a caching system to utilize a snapshot file or other limited size data structure to store a portion of the data stored in a cache. The snapshot file can be stored on persistent or otherwise non-transitory storage so that, even in case of a restart, crash or power loss event, the data stored in the snapshot file persists and can be used by the caching system after starting up. The snapshot file can then be used to restore at least some data into the cache in cases where the cached data in the cache is lost. For example, in cases of a cold-start or restart, the caching system can load data from the snapshot file into the empty cache. This can increase the number of cache hits since the cache is repopulated with useful data at startup.

Abstract: Systems and methods are directed to an eventually consistent replicated data store that uses, for its underlying storage, a computer software library that provides a high-performance embedded database for data. The replicated data store employs a plurality of hosts interconnected to one another, allowing for writes to any host and full awareness of membership across all hosts. With the data replication framework, various modes are allowed to be built up on top of the core system.

Abstract: Processes are disclosed for decreasing contention in caches in order to increase the efficiency of multi-threaded or multi-processor systems. By using multiple locks in a cache, smaller portions of the cache can be locked during cache updates (e.g., during a data update or a storage block eviction). As only small portions of the cache are locked at any given time, contention between threads, particularly in multi-processor implementations, will likely be reduced. For example, if different threads are trying to update different entries in the cache, the threads can proceed with updating the cache concurrently.

Abstract: The disclosure describes embodiments of a distributed caching system that are configured to store handshake data between client devices and servers, enabling handshake transaction to be resumed in case of interruption. Client devices can resume the handshake transaction even if assigned to new servers as the new servers can obtain the handshake data identifiers from the distributed caching system.

Abstract: The disclosure describes embodiments of a distributed caching system that are configured to store session state identifiers in a networked cache, enabling dynamic allocation of requests to servers. Client devices can resume secure sessions even if assigned to new servers as the new servers can obtain the session state identifiers from the distributed caching system. In at least some cases, the client device can be authenticated without the server having to perform a full authentication, thereby reducing the workload of the server and decreasing latency as the server can respond faster.

Abstract: Processes and systems are disclosed for partitioning a number of producer systems among a number of leasing agents. A leasing agent can identify a set of available producer systems within an environment and can partition the producer systems among each of the number of leasing agents. The leasing agent can then inform the other leasing agents of the producer systems assigned to each of the leasing agents. As the number of available producer systems change, the partitioning process can be repeated to rebalance the assignment of producer systems among the leasing agents.

Abstract: Processes and systems are disclosed for leasing a consumer system access to a producer system. The consumer system can communicate with a set of leasing agents to obtain the identity of a number of producer systems capable of providing the consumer system with access to a service. Each leasing agent may provide the consumer system with access to a producer system for a period of time. If the consumer system requires further access to a particular producer system, the consumer system can request renewal of the lease from the leasing agent that provided the initial lease of the producer system to the consumer system.

Abstract: Processes and systems are disclosed for leasing a producer virtual machine on behalf of a consumer virtual machine in an overlay network. The consumer host of the consumer virtual machine can communicate with a set of leasing agents to obtain the identity of a number of producer virtual machines capable of providing the consumer virtual machine with access to a service. When the consumer virtual machine attempts to communicate with a producer system, the consumer host can identify a producer host that hosts a target producer virtual machine and redirect a service request to the producer host.

Abstract: Processes and systems are disclosed for selecting a producer system from a number of producer systems to lease to a consumer system. A leasing agent, in response to a request from the consumer system for access to a service at a producer system, can identify a producer system to lease to the lease requestor based, at least in part, on a selection weight associated with each producer system that the leasing agent is assigned. The selection weights can be modified based on status information associated with each of the producer systems. This status information may be obtain from the producer systems and/or from a consumer system that has previously accessed the producer system. The consumer system may provide the status information to the leasing agent as part of the consumer system's lease request.

Abstract: Generally described, the present disclosure is directed to an eventually consistent replicated data store that uses, for its underlying storage, a computer software library that provides a high-performance embedded database for data. The replicated data store employs a plurality of hosts interconnected to one another, allowing for writes to any host and full awareness of membership across all hosts. With the data replication framework disclosed herein, various modes are allowed to be built up on top of the core system.

Abstract: An eventually consistent replicated data store uses, for its underlying storage, a computer software library that provides a high-performance embedded database for data. The replicated data store employs a plurality of hosts interconnected to one another, allowing for writes to any host and full awareness of membership across all hosts. With such a data replication framework, various modes are allowed to be built up on top of the core system.

Abstract: Management of a data cache having a compressed portion and an uncompressed portion by adaptively and dynamically allocating the relative amount space each portion receives. The relative sizes are defined based on one or more cost metrics and benefit metrics. The metrics are selected based on the performance of an application utilizing the cache. An optimized benefit relative to the cost is defined. Application operations on the cache are sampled and the relative cost and benefit is determined for different ratios of uncompressed to compressed cache sizes. The size of the uncompressed portion relative to the compressed portion is then re-adjusted based on an optimal cost-to-benefit ratio for the application.