Abstract: Disclosed is a system for storage management comprising: receiving capacity information within a predetermined time period from a plurality of storage in a first appliance pool and a second appliance pool, each having a plurality of storage systems; tracking a first capacity limit of the first storage system in the first appliance pool based on the capacity information; performing a first capacity prediction of the first storage system based on the tacked result of the first storage system, wherein the first capacity prediction indicating when the first storage system will be out of space based on the capacity information and the tracked result; generating a first recommendation indicating when an extra capacity of the first storage system should be added based on one or more attributes including the first capacity prediction; and presenting the first recommendation with one or more options.

Abstract: Described is a system for detecting corruption in a deduplicated object storage system accessible by one or more microservices while minimizing costly read operations on objects. A similarity group verification path is selected based upon detection of an object storage memory size condition. The similarity group verification path is performed by one or more worker nodes. The similarity group verification path includes worker node phases that verify whether objects have been corrupted without having to incur costly read operations.

Abstract: Object service receives request to copy file to destination and identifies group identifier for fingerprints group corresponding to sequential segments in file. Object service communicates request for fingerprints group to deduplication service associated with group identifier range including group identifier. Deduplication service communicates fingerprints group, retrieved from fingerprint storage, to object service, which communicates fingerprints group and group identifier to destination. Object service communicates request for file segments, corresponding to fingerprints missing in destination, communicated from destination, to deduplication service, which communicates requested segments, retrieved from source storage, to object service, which communicates requested segments to destination.

Abstract: Systems and methods for performing data protection operations including garbage collection operations and copy forward operations. For deduplicated data stored in a cloud-based storage or in a cloud tier that stores containers containing dead and live segments or dead and live regions such as compression regions, the dead compression regions are deleted by copying the live compression regions into new containers and then deleting the old containers. The copy forward is based on a recipe from a data protection system and is performed using a serverless approach.

Abstract: Systems and methods for performing data protection operations including garbage collection operations and copy forward operations. For deduplicated data stored in a cloud-based storage or in a cloud tier that stores containers containing dead and live regions such as compression regions, the dead segments in the dead compression regions are deleted by copying the live compression regions into new containers and then deleting the old containers. The copy forward is based on a recipe from a data protection system and is performed using a microservices based approach.

Abstract: Systems and methods enabling garbage collection operations and normal system operations concurrently. Concurrent operations are performed by configuring a similarity group to permit garbage collection and normal operations. This may include creating a new subgroup in a similarity group for write and deduplication purposes such that an impacted subgroup can be cleaned.

Abstract: One example method includes receiving a write request that includes a data structure version to be written, wherein the data structure version is associated with a unique identifier, storing the data structure version in association with the unique identifier, receiving a read request for a most recent version of the data structure and, when the stored data structure version is not the most recent version of the data structure, examining respective unique identifiers of each of a group of other stored data structure versions to determine which stored data structure version is the most recent. Finally, the example method includes returning the most recent data structure version, notwithstanding that one or more other data structure versions existed at the time that the read request was received.

Abstract: Embodiments of the present disclosure relate to a method, a system, and a non-transitory machine-readable medium for assigning performance defects to software processing components. Provided is a method that receives performance data of a set of performance runs of a software application across a set of software processing components of a file software system, where the performance data comprises multiple labeled performance features that are associated with the set of software processing components; receives, from a software processing component expert, a user-selection of a subset of labeled performance features of the several labeled performance features; and trains a machine learning (ML) model to determine whether one or more of the set of software processing components is associated with performance regressions of the software application, using the subset of labeled performance features of the performance data as training data.

Abstract: A controller is provided to manage searching a storage system. A search query including a search term is received. The controller generates a set of tasks associated with conducting a search of the storage system for any objects in the storage system having the search term. These tasks are pushed onto a message queue. A set of workers are allocated to retrieve and process the tasks. A worker processes a task in parallel with another worker processing another task.

Abstract: A deduplicated file system includes a set of microservices including front-ends and back-ends. Assignments of files are balanced across front-ends. The files are represented by segment trees including multiple segment levels. Assignments of similarity groups are balanced across back-ends. Similarity groups are associated with segments at a lower-level of the segment trees that form the files. Front-ends are responsible for operations involving an upper-level of the trees. Back-ends are responsible for operations involving the lower-level of the trees. A mapping of file assignments to front-ends and of similarity group assignments to back-ends is stored. A request to perform a file system operation is received. The mapping is consulted to identify particular front and back-ends that should be responsible for handling and processing the request.

Abstract: Disclosed is a system for storage management comprising: receiving capacity information within a predetermined time period from a plurality of storage in a first appliance pool and a second appliance pool, each having a plurality of storage systems; tracking a first capacity limit of the first storage system in the first compliance pool based on the capacity information; performing a first capacity prediction of the first storage system based on the tacked result of the first storage system, wherein the first capacity prediction indicating when the first storage system will be out of space based on the capacity information and the tracked result; generating a first recommendation indicating when an extra capacity of the first storage system should be added based on one or more attributes including the first capacity prediction; and presenting the first recommendation with one or more options.

Abstract: Described is a system for a providing a service (or microservice) for performing deduplication for an object storage. The service (or microservice) may be source-agnostic in that it may receive data from multiple types of source systems by providing a uniform set of functions for deduplicating and writing the data to a destination object storage. The set of functions encapsulate a previously dispersed set of functionality provided by various components. Accordingly, the service provides a single scalable and stateless component for performing deduplication. For example, the service (e.g. deduplication service) may receive object related information and perform a filtering to accelerate network transfers. Accordingly, the service provides the ability to only transfer and write data that does not already exist on the object storage.

Abstract: Systems and methods for estimating the number of workers needed to perform a garbage collection operation are disclosed. Similarity groups are used to identify segments associated with objects in a computing system. Using deletion records that identify objects to be deleted, the similarity groups impacted by the deletion records can be identified. The number of workers can be determined based on the impacted similarity groups. More specifically, the number of impacted similarity groups and/or workers can be evaluated in terms of memory requirements, input/output constraints and/or time requirements to estimate the number or workers needed to clean similarity groups impacted by a garbage collection operation.

Abstract: Described is a system for detecting corruption in a deduplicated object storage system accessible by one or more microservices while minimizing costly read operations on objects. A testing tool inserts object corruptions in object storage. A verification path is performed by a controller module and one or more worker nodes. The testing tool verifies whether the executed verification path accurately detected the testing tool's inserted object corruptions.

Abstract: Systems and methods for marking similarity groups impacted by a garbage collection operation are disclosed. Similarity groups are used to identify segments associated with objects in a computing system. Using deletion records that identify objects to be deleted, the similarity groups impacted by the deletion records can be identified. The live segments associated with the impacted similarity groups are also identified. This allows segments that are associated with the deleted objects and that are not associated with any live objects to be removed.

Abstract: A method, computer program product, and computing system for receiving a candidate data portion; calculating a distance-preserving hash for the candidate data portion; and performing an entropy analysis on the distance-preserving hash to generate a hash entropy for the candidate data portion.

Abstract: A source worker node at a source site fetches a task from a message queue. The task specifies replicating a first object at the source site to a destination site. A request for a connection is issued from the source worker node to the destination site. The request is received by a load balancer at the destination site and assigned to a destination worker node. A connection is established between the source and destination worker nodes. A determination is made that the destination site does not include an object that is the same as the first object. Upon the determination, a deduplication is performed between the source and destination worker nodes of segments into which the first object has been divided. Deduplicated segments of the first object are transmitted from the source worker node to the destination worker node for storage at the destination site.

Abstract: Described is a system for detecting corruption in a deduplicated object storage system accessible by one or more microservices while minimizing costly read operations on objects. A controller module and one or more worker nodes execute verification paths in concert with each other to identify object corruptions. The controller module estimates a number of worker nodes required for execution of different types of verification paths.

Abstract: Described is a system for a verification microservice engine for generating and deploying a controller module and one or more worker nodes to detect corruption in a deduplicated object storage system accessible by one or more microservices while minimizing costly read operations on objects. A worker node(s) stores slice recipes into worker node memory that correspond to slice recipe names verified by a controller module. For each slice recipe, the worker node(s) detects whether slice recipe corruptions, similarity group corruptions and/or compression region corruptions exist.

Abstract: A schedule is stored indicating a frequency of replication from source to destination sites. When a replication job is initiated, information identifying one or more objects at the source site to be replicated is copied into a snapshot without pausing user operations against the one or more objects. The snapshot is compared with a previous snapshot to generate replication tasks for the replication job. The replication tasks are placed onto a message queue at the source site, where a worker node at the source site retrieves a replication task from the message queue and processes the replication task in conjunction with a worker node at the destination site.