Abstract: Data migrated from a deduplicated storage appliance are stored as cloud units at cloud storage. Each cloud unit includes containers including data containers storing segments of files, segment tree containers storing upper-level segments of segment trees representing the files, and cloud containers storing headers from the data and segment tree containers. A header for a data container includes fingerprints identifying the segments of files. A header for a segment tree container includes fingerprints identifying the upper-level segments. The cloud units are maintained in different states of accessibility including read-write, read-only, and offline. Upon detecting that local storage of the appliance does not have space to support a cloud unit in a read-write or read-only state, another cloud unit that is in a first state is selected, the first state being the read-write or read-only state. The selected cloud unit is placed in a second state, different from the first state.

Abstract: Cloud units in cloud storage include containers including data containers storing file segments, segment tree containers storing upper-level segments of segment trees representing the files, and cloud containers storing headers from the data and segment tree containers. A header for a data container includes fingerprints identifying the file segments. A header for a segment tree container includes fingerprints identifying the upper-level segments. A storage appliance stores a first level of metadata for each cloud unit in a read-write state, a second level of metadata for each cloud unit in a read-only state, and a third level of metadata for each cloud unit in an offline state. The third level requires less storage than the first and second levels. The second level requires less storage than the first level. The limited metadata space of the appliance is managed by maintaining at least a subset of cloud units in the read-only state.

Abstract: Containers are generated including data containers storing segments of files, segment tree containers storing upper-level segments of segment trees representing the files, and cloud containers storing headers from the data and segment tree containers. A header for a data container includes fingerprints identifying the segments of files. A header for a segment tree container includes fingerprints identifying the upper-level segments. The containers are sent to cloud storage for storage as a cloud unit. The cloud unit is attached to a deduplicated storage appliance by storing at the appliance the segment tree and cloud containers, cloud configuration details, and an index. The cloud configuration details identify the cloud storage having the cloud unit. The index maps the fingerprints to the containers. When a cloud unit is to be detached, the segment tree and cloud containers, and index from the local storage are deleted, but the cloud configuration details are maintained.

Abstract: Resources of a deduplicated storage appliance are monitored to generate a consumption matrix indicating resource utilization. A threshold map associated with a subsystem is accessed. The subsystem is responsible for an operation involving files stored in the appliance. The threshold map specifies resources required by the subsystem for the operation and thresholds below which multi-streaming for the operation is allowed. An assessment is made as to whether the operation should be handled by multi-streaming. The assessment includes determining whether the resources are underutilized or overutilized based on the consumption matrix and the threshold map. When the resources are overutilized, multi-streaming is not triggered. When the resources are underutilized, multi-streaming is triggered. Periodic reassessments are made to determine whether the operation should be handled by multi-streaming.

Abstract: Prioritization in cloud migration for deduplication systems is described. A system creates a set of priorities corresponding to backup files in a set of backup files, wherein each priority is based on estimating a time required for a corresponding backup file to migrate from a local storage system to a cloud storage system, and/or an amount of data deduplicated when the corresponding backup file is stored. The system copies a backup file, which corresponds to the highest priority in the set of priorities, from the set of backup files stored in the local storage system to a cloud storage system. Completing the copying of the set of backup files to the cloud storage system, the system copies a backup file, which corresponds to the lowest priority in the set of priorities, from the set of backup files stored in the local storage system to a cloud storage system.

Abstract: Described is a system (and method) for efficient metadata verification when leveraging multi-streams as part of data deduplication system. The system may efficiently perform metadata integrity verification by dynamically invoking multi-streaming to leverage parallelization for metadata verification. Accordingly, the system may better utilize bandwidth and processing capabilities to reduce verification time.

Abstract: Described is a system (and method) for efficient metadata verification when leveraging multi-streams as part of data deduplication system. The system may efficiently perform metadata integrity verification by dynamically invoking multi-streaming to leverage parallelization for metadata verification. Accordingly, the system may better utilize bandwidth and processing capabilities to reduce verification time.