Abstract: Described is a system for leveraging incremental backup metadata of a deduplication storage system to perform an efficient data restore. The system may reference metadata stored as part of incremental backup infrastructure to determine only the changes to the data that occurred subsequent to the restore point. As a result, the system may reduce data read processing by retrieving only the changed data from the deduplication storage system. Moreover, the system may reduce the required bandwidth for the restore by providing only the changed data to a target device of the restore. Data retained by the client system such as the current version of the data may then be overwritten with the reduced set of data to complete the restore. As a result, the system may perform an efficient restore by reducing read accesses and bandwidth requirements, and thus, improve overall restore performance.

Abstract: Embodiments of a system and method to track the locality of a file being restored at the time of prefetching; and a mechanism to dynamically adjust the prefetching parallelism, per read batch, optimally based on the locality and other heuristics, such as system load. A process tracks locality of data elements in a batched data stream, as corresponds to a number of different container IDs accessed by the data elements. The prefetch nominally works serially on the data elements, however, if the locality exceeds a threshold separating acceptable versus non-acceptable distribution of data accesses, each batch is divided into a number of smaller sub-batches that are then pre-fetched in parallel with one another.

Abstract: During a current backup of a file system of a client, a determination is that a file in the current backup includes first data regions found in a file from a previous backup, and second data regions that are new. A virtual synthetic file is generated by writing the second data regions, and synthesizing the first data regions by referencing the file from the previous backup. A request received to restore the particular file includes identifications of other files still present at client. A determination is that the particular file corresponds to the virtual synthetic file and that the other files still present at client include the file from the previous backup. The second data regions are read to send to client. A complete version of the particular file is assembled at client using the file from the previous backup, still present at client, and the second data regions.

Abstract: A request is received to restore a file at a deduplicated storage system to a client. The file resides at the storage system as a synthetic file based on a base file at the storage system. The request includes an indication that the base file is also present at the client. Metadata generated during a backup of the file to the storage system is reviewed. The metadata includes references to data determined to be in the base file at the storage system, and references to other data determined to not be in the base file at the storage system. The other data determined to not be in the base file is read from the storage system and transmitted to the client. Upon receipt, the client assembles the requested file using the base file present at the client and the other data determined to not be in the base file.

Abstract: Embodiments of a system and method to track the locality of a file being restored at the time of prefetching; and a mechanism to dynamically adjust the prefetching parallelism, per read batch, optimally based on the locality and other heuristics, such as system load. A process tracks locality of data elements in a batched data stream, as corresponds to a number of different container IDs accessed by the data elements. The prefetch nominally works serially on the data elements, however, if the locality exceeds a threshold separating acceptable versus non-acceptable distribution of data accesses, each batch is divided into a number of smaller sub-batches that are then pre-fetched in parallel with one another.

Abstract: A request is received to restore a file at a deduplicated storage system to a client. The file resides at the storage system as a synthetic file based on a base file at the storage system. The request includes an indication that the base file is also present at the client. Metadata generated during a backup of the file to the storage system is reviewed. The metadata includes references to data determined to be in the base file at the storage system, and references to other data determined to not be in the base file at the storage system. The other data determined to not be in the base file is read from the storage system and transmitted to the client. Upon receipt, the client assembles the requested file using the base file present at the client and the other data determined to not be in the base file.

Abstract: Described is a system for leveraging incremental backup metadata of a deduplication storage system to perform an efficient data restore. The system may reference metadata stored as part of incremental backup infrastructure to determine only the changes to the data that occurred subsequent to the restore point. As a result, the system may reduce data read processing by retrieving only the changed data from the deduplication storage system. Moreover, the system may reduce the required bandwidth for the restore by providing only the changed data to a target device of the restore. Data retained by the client system such as the current version of the data may then be overwritten with the reduced set of data to complete the restore. As a result, the system may perform an efficient restore by reducing read accesses and bandwidth requirements, and thus, improve overall restore performance.

Abstract: Embodiments of an instant recall process and system for long-term data stored on a cloud storage tier. Embodiments include saving a content handle of a file in a cloud storage tier as an extended attribute in a single file system namespace; moving the file from the cloud storage tier to an active storage tier for data processing; recalling the file from the active storage tier to the cloud storage tier upon completion of the data processing; using the content handle from hidden metadata for a working copy of the file; and saving a hash of a segment reference as part of the extended attribute.

Abstract: A method and system for storage load balancing based on virtual synthetics metadata. When storing data onto a storage cluster, data submitted thereto may often be distributed unevenly across the constituent storage nodes thereof. To address the issue, some form of load balancing (or re-distribution of data) across the storage nodes may be implemented. Existing load balancing techniques, however, tend to migrate data between storage nodes without consideration for the efficient utilization of available storage space on the storage node where the data ends up (or destination storage node). Accordingly, the disclosed method and system propose a load balancing mechanism whereby the migrated data dedupes well, thereby securing the efficient consumption of storage space on the destination storage node.

Abstract: A method and system for storage load balancing based on virtual synthetics metadata. When storing data onto a storage cluster, data submitted thereto may often be distributed unevenly across the constituent storage nodes thereof. To address the issue, some form of load balancing (or re-distribution of data) across the storage nodes may be implemented. Existing load balancing techniques, however, tend to migrate data between storage nodes without consideration for the efficient utilization of available storage space on the storage node where the data ends up (or destination storage node). Accordingly, the disclosed method and system propose a load balancing mechanism whereby the migrated data dedupes well, thereby securing the efficient consumption of storage space on the destination storage node.

Abstract: According to various embodiments, systems and methods are provided that relate to shared access to Storage Area Networks (SAN) devices. In one embodiment, a Storage Area Network (SAN) host is provided, comprising: a server component: a first host bus adapter configured to be connected to a SAN client over a first SAN; a second host bus adapter configured to be connected to a SAN storage device over a second SAN; and wherein the server component is configured to manage a data block on the SAN storage device, receive a storage operation request from the SAN client through the first host bus adapter, and in response to the storage operation request, perform a storage operation on the data block, the storage operation being performed over the second SAN through the second host bus adapter.

Abstract: A method for managing backups includes receiving, via a first backup stream, first data associated with a first tagged backup stripe, wherein the first tagged backup stripe is associated with a first routing tag, directing, based on the first routing tag, the first data to a first backup instance, receiving, via a second backup stream, second data associated with a second tagged backup stripe, wherein the second tagged backup stripe is associated with the first routing tag, directing, based on the first routing tag, the second data to the first backup instance, and performing, in the first backup instance, a deduplication operation on the first data and the second data.

Abstract: Embodiments are described for performing file restores from remote high-latency storage tiers by reading available data from a local low-latency tier in a deduplication appliance. A request to restore a previously segmented and deduplicated file can be received by a storage appliance from an application, each segment having a fingerprint. The name of the file can be looked up in an index on the storage appliance, and a first batch of fingerprints of segments of the file can be retrieved from the index. Each fingerprint can be looked up in metadata in the index to determine whether the segment corresponding to the fingerprint is available locally and therefore need not be retrieved locally. A list of local and remote prefetch segments is generated, and a prefetch request is generated for each list, if non-empty. Use of the prefetch scheme can be dynamically turned on or off.

Abstract: According to various embodiments, systems and methods are provided that relate to shared access to Storage Area Networks (SAN) devices. In one embodiment, a Storage Area Network (SAN) host is provided, comprising: a server component: a first host bus adapter configured to be connected to a SAN client over a first SAN; a second host bus adapter configured to be connected to a SAN storage device over a second SAN; and wherein the server component is configured to manage a data block on the SAN storage device, receive a storage operation request from the SAN client through the first host bus adapter, and in response to the storage operation request, perform a storage operation on the data block, the storage operation being performed over the second SAN through the second host bus adapter.

Abstract: Embodiments are described for performing a storage tier-specific file system operation in a file system that comprises a namespace that spans multiple tiers of storage. A file system command can be a tier-specific deletion of files in a multi-tier file system, or a move data operation from a first tier of storage to a second tier of storage, affecting both the first and second tier of storage. The file system command can generate a copy of the namespace for the first tier of storage and a snapshot of the data of the first tier of storage referenced in the file system operation. Functionality may be performed using one or more storage appliances, each of which can have its open processing system, memory and storage device(s). Storage tiers can include active tier, cloud tier, and remote or archive storage tier.

Abstract: A method for managing backups includes receiving, via a first backup stream, first data associated with a first tagged backup stripe, wherein the first tagged backup stripe is associated with a first routing tag, directing, based on the first routing tag, the first data to a first backup instance, receiving, via a second backup stream, second data associated with a second tagged backup stripe, wherein the second tagged backup stripe is associated with the first routing tag, directing, based on the first routing tag, the second data to the first backup instance, and performing, in the first backup instance, a deduplication operation on the first data and the second data.

Abstract: Embodiments of an instant recall process and system for long-term data stored on a cloud storage tier. Embodiments include saving a content handle of a file in a cloud storage tier as an extended attribute in a single file system namespace; moving the file from the cloud storage tier to an active storage tier for data processing; recalling the file from the active storage tier to the cloud storage tier upon completion of the data processing; using the content handle from hidden metadata for a working copy of the file; and saving a hash of a segment reference as part of the extended attribute.

Abstract: Embodiments are described for reducing memory usage and disk I/O's for reading and writing streams in a deduplication storage system. Generations of backups of a stream can be tracked for the stream. In the first generation (first backup), stream data is ingested, segmented, and written to disk without looking up a fingerprint of each segment, to save disk I/O's and memory. After the first generation, an amount of stream data processed is tracked, and at dynamically tunable increments of data processed, a fingerprint index lookup is performed to disk. The fingerprint index is read into memory and subsequent fingerprint lookups are preformed to cache, until a next dynamically tunable fingerprint lookup to disk. The dynamically tunable increment of processed data can be randomly reset.

Abstract: Embodiments are described for performing file restores from remote high-latency storage tiers by reading available data from a local low-latency tier in a deduplication appliance. A request to restore a previously segmented and deduplicated file can be received by a storage appliance from an application, each segment having a fingerprint. The name of the file can be looked up in an index on the storage appliance, and a first batch of fingerprints of segments of the file can be retrieved from the index. Each fingerprint can be looked up in metadata in the index to determine whether the segment corresponding to the fingerprint is available locally and therefore need not be retrieved locally. A list of local and remote prefetch segments is generated, and a prefetch request is generated for each list, if non-empty. Use of the prefetch scheme can be dynamically turned on or off.

Abstract: A data management device includes a persistent storage and a processor. The persistent storage includes a local object storage. The local object storage includes local data objects, local meta-data objects, and remote meta-data objects. The processor segments a file into file segments, deduplicates the file segments, stores the deduplicated file segments in a remote data object of a remote object storage, and stores meta-data of the deduplicated file segments in a remote meta-data object of the remote meta-data objects.