Abstract: A deduplicated microservices-based storage system and method running in a cloud includes frontend and backend microservice nodes that process requests from a plurality of clients and interface with a master node and one or more worker nodes connected as a network for deduplicating, storing and retrieving data from object storage. A custom metric compares demand on system resources to thresholds and dynamically scales resources to add or eliminate resources according to demand. The master node partitions the range of values identifying data similarity groups across the worker nodes, reassigns ranges of values using consistent hashing as worker nodes are scaled, and maintains a mapping of ranges of values to worker IP addresses.

Abstract: In a method of processing data similarity groups, a combined read/write cache comprises an in-memory data structure having a fixed allocation of physical memory that includes a write portion comprising memory allocated to write entries and a read portion comprising memory allocated to read entries. Similarity group entries are written into the cache using keys based upon an identifier of a similarity group, and similarity group entries are read from the cache using keys based upon both similarity group and subgroup identifiers. The sizes of memory allocated to the write portion and to the read portion are dynamically varied within the fixed memory allocation based upon demand while maintaining the fixed memory allocation constant.

Abstract: A deduplicated microservices-based storage system and method running in a cloud includes frontend and backend microservice nodes that process requests from a plurality of clients and interface with a master node and one or more worker nodes connected as a network for deduplicating, storing and retrieving data from object storage. A custom metric compares demand on system resources to thresholds and dynamically scales resources to add or eliminate resources according to demand. The master node partitions the range of values identifying data similarity groups across the worker nodes, reassigns ranges of values using consistent hashing as worker nodes are scaled, and maintains a mapping of ranges of values to worker IP addresses.