Abstract: Systems and methods are disclosed for managing deduplicated images of data objects that change over time. The method includes: organizing unique content of each data object as a plurality of content segments and storing the content segments in a data store; for each data object, creating an organized arrangement of hash structures, wherein each structure, for a subset of the hash structures, includes a hash signature for a corresponding content segment and is associated with a reference to the corresponding content segment, and for each data object, maintaining an organized arrangement of temporal structures to represent a corresponding data object over time, wherein each structure is associated with a temporal state of the data object, and wherein each temporal state is associated with the hash structures representing the content of the data object during that temporal state.

Abstract: Systems and methods are disclosed for copying a data object to a target storage pool using a hybrid of storage pools, in which at least one of the storage pools of the hybrid is particularly efficient at identifying data that should be used for copying the data object to the target storage pool, and at least one of the storage pools of the hybrid is particularly efficient at retrieving the data that should be sent to the target storage pool. The system comprises a performance storage pool for storing data and having relatively high performance for retrieving stored data; a deduplicating storage pool for storing deduplicated data and storing metadata about data objects in the system and which has relatively high performance for identifying and specifying differences in a data object over time; and a controller for causing the performance storage pool to retrieve differences and provide the data to the target storage pool.

Abstract: Systems and methods for checking data integrity of a data object copied between storage pools in a storage system by comparing data samples copied from data objects. A series of successive copy operations are scheduled over time for copying a data object from a source data store to a target data store. A first data sample is generated based on a sampling scheme comprising an offset and a period. A second data sample is generated using a similar sampling scheme. The blocks of data in the first data sample and the second data sample are compared to determine if they differ to thereby indicate that the data object at the target store differs from the corresponding data object at the source data store.

Abstract: Systems and methods are disclosed for checking the data integrity of a data object copied between storage pools in a storage system by comparing data fingerprints of data objects, by scheduling a series of successive copy operations over time for copying a data object from a source data store to a target data store; generating a partial fingerprint of the data object at the source data store using a data fingerprinting operation that creates a fingerprint from a subset of data of the data object; sending the partial fingerprint of the data object to the target data store; sending any new data contents for the data object to the target data store; and creating a partial fingerprint of the data object at the target data store and comparing it to the partial fingerprint sent to the target data store to determine if they differ.

Abstract: Described herein are systems and methods for garbage collection prediction. A temporal graph is received, the temporal graph including nodes, the nodes including hash references to objects. An accumulated difference count is updated when a node is added to the temporal graph, the accumulated difference count including a number of hash differences between a parent node and its children nodes in the temporal graph. A divested difference count is updated when a node is removed from the temporal graph, the divested difference count including a number of hash differences referenced by the removed node but not by either a parent node of the removed node or any child nodes of the removed node. The outcome of the garbage collection is predicted based on at least one of the accumulated difference count and the divested difference count.

Abstract: Described herein are systems and methods for efficient data replication. A set of hashes for a source object to be replicated is sent from the source local deduplication store to the remote server. The remote server generates a set of object hashes representative of data in the source object that is already present on the remote server, and data indicative of source object hashes that are not present on the remote server. The remote server transmits the generated data to the source local deduplication store. The source local deduplication store identifies portions of the source object that are not already present on the remote server based on the received data. The source local deduplication store transmits the identified portions of the source object to the remote server to replicate the source object on the remote server.

Abstract: Systems and methods are provided for caching hashes for deduplicated data. A request to read data from the deduplication data store is received. A persist header stored in a deduplication data store is identified in a first hash structure that is not stored in memory of the computing device. The persist header comprises a set of hashes that includes a hash that is indicative of the data the computing device requested to read. Each hash in the set of hashes represents data stored in the deduplication data store after the persist header that is co-located with other data represented by the remaining hashes in the set of hashes. The set of hashes is cached in a second hash structure stored in the memory, whereby the computing device can identify the additional data using the second hash structure if the additional data is represented by the persist header.

Abstract: Systems and methods are disclosed for copying a data object to a target storage pool using a hybrid of storage pools, in which at least one of the storage pools of the hybrid is particularly efficient at identifying data that should be used for copying the data object to the target storage pool, and at least one of the storage pools of the hybrid is particularly efficient at retrieving the data that should be sent to the target storage pool. The system comprises a performance storage pool for storing data and having relatively high performance for retrieving stored data; a deduplicating storage pool for storing deduplicated data and storing metadata about data objects in the system and which has relatively high performance for identifying and specifying differences in a data object over time; and a controller for causing the performance storage pool to retrieve differences and provide the data to the target storage pool.

Abstract: Systems and methods are disclosed for forming deduplicated images of a data object that changes over time using difference information between temporal states of the data object. The method includes organizing the content of the data object for a first temporal state as a plurality of content segments and storing the content segments in a data store; creating an organized arrangement of hash structures to represent the data object in its first temporal state; receiving difference information for the data object; forming at least one hash signature for the changed content; and storing the changed content that is unique in the data store as content segments. The method also includes determining, subsequent to receiving the changed content at the deduplicating content store, whether the changed content should be stored by searching for the hash signature for the changed higher-level hash structure in the global cache of the deduplicating content store.

Abstract: Systems and methods are disclosed for using a first deduplicating store to update a second deduplicating store with information representing how data objects change over time, said method including: at a first and a second deduplicating store, for each data object, maintaining an organized arrangement of temporal structures to represent a corresponding data object over time, wherein each structure is associated with a temporal state of the data object and wherein the logical arrangement of structures is indicative of the changing temporal states of the data object; finding a temporal state that is common to and in temporal proximity to the current state of the first and second deduplicating stores; and compiling and sending a set of hash signatures for the content that has changed from the common state to the current temporal state of the first deduplicating store.

Abstract: Systems and methods are provided for an asynchronous data replication system in which the remote replication reduces bandwidth requirements by copying deduplicated differences in business data from a local storage site to a remote, backup storage site, the system comprising: a local performance storage pool for storing data; a local deduplicating storage pool for storing deduplicated data, said local deduplicating storage pool further storing metadata about data objects in the system and which has metadata analysis logic for identifying and specifying differences in a data object over time; a remote performance storage pool for storing a copy of said data, available for immediate use as a backup copy of said data to provide business continuity to said data; a remote deduplicating storage pool for storing deduplicated data; and a controller for synchronizing the remote performance storage pool to have the second version of the data object using deduplicated data.

Abstract: Systems and methods are disclosed for copying a data object to a target storage pool using a hybrid of storage pools, in which at least one of the storage pools is particularly efficient at identifying data that should be used for copying the data object to the target storage pool, and at least one of the storage pools is particularly efficient at retrieving the data that should be sent to the target storage pool. The system comprises a performance storage pool for storing data and having relatively high performance for retrieving stored data; a deduplicating storage pool for storing deduplicated data and storing metadata about data objects in the system and which has relatively high performance for identifying and specifying differences in a data object over time; and a controller for causing the performance storage pool to retrieve differences and provide the data to the target storage pool.

Abstract: Techniques are disclosed for providing content data storage services to a remote device over the internet to enable access of the remote device in the cloud. A content data storage device receives data indicative of a subscription to content data storage services from a remote device. The content data storage device provisions cloud storage to provide the content data storage services subscribed to by the remote device. Data associated with the remote device is replicated to the provisioned cloud storage to provide a replicated device in the cloud. Data indicative of a request to use the replicated device in the cloud is received. The content data storage device executes the replicated device in the cloud, thereby providing access of the remote device in the cloud for the remote device.

Abstract: Techniques are disclosed for remotely backing up data associated with a plurality of storage environments. A first computing device receives a storage type associated with a second computing device managed by the first computing device. Storage parameters are configured based on the storage type to customize a backup process for the second computing device based on the storage type. Data associated with the second computing device is protected using the storage parameters, wherein protecting data associated with the second computing device further includes copying at a first point in time a full copy of data associated with the second computing device, and copying changes to the data associated with the second computing device at a set of points in time later than the first point in time, the set of points in time being based on an end-user policy.

Abstract: Techniques are disclosed for creating, in a network, a single instance of deduplicated data across a plurality of end user data. A first computing device receives data associated with a plurality of computing devices, the plurality of computing devices being managed by the first computing device. The first computing device aggregates and deduplicates the data associated with each of the plurality of computing devices. The deduplicated aggregated data set is then transmitted to a second computing device for further aggregation and deduplication with one or more additional aggregated data sets generated by other computing devices managing respective sets of computing devices.

Abstract: Systems and methods are disclosed for checking the data integrity of a data object copied between storage pools in a storage system by comparing data fingerprints of data objects, by scheduling a series of successive copy operations over time for copying a data object from a source data store to a target data store; generating a partial fingerprint of the data object at the source data store using a data fingerprinting operation that creates a fingerprint from a subset of data of the data object; sending the partial fingerprint of the data object to the target data store; sending any new data contents for the data object to the target data store; and creating a partial fingerprint of the data object at the target data store and comparing it to the partial fingerprint sent to the target data store to determine if they differ.

Abstract: Systems and methods for estimating data similarity between an inserted volume of data and a stored volume of data during file backup of a deduplicated data store when the ancestry of the inserted data to previously-stored data is unknown to identify an ancestor of the inserted volume of data in the stored volume so that only incremental data of the inserted volume is stored, the systems and methods comprising ingesting a volume of data, creating a subset of bits for the ingested volume using a filtering process, creating a subset of bits for each volume of stored data using the filtering process, comparing the subset of bits for the ingested volume with the subset of bits for each of the stored volumes, and determining the subset of bits for a stored volume with the most bits in common with the subset of bits for the ingested volume.

Abstract: Systems and methods are disclosed for forming deduplicated images of a data object that changes over time using difference information between temporal states of the data object. The method includes organizing the content of the data object for a first temporal state as a plurality of content segments and storing the content segments in a data store; creating an organized arrangement of hash structures to represent the data object in its first temporal state; receiving difference information for the data object; forming at least one hash signature for the changed content; and storing the changed content that is unique in the data store as content segments. The method also includes determining, subsequent to receiving the changed content at the deduplicating content store, whether the changed content should be stored by searching for the hash signature for the changed higher-level hash structure in the global cache of the deduplicating content store.

Abstract: Systems and methods are provided for an asynchronous data replication system in which the remote replication reduces bandwidth requirements by copying deduplicated differences in business data from a local storage site to a remote, backup storage site, the system comprising: a local performance storage pool for storing data; a local deduplicating storage pool for storing deduplicated data, said local deduplicating storage pool further storing metadata about data objects in the system and which has metadata analysis logic for identifying and specifying differences in a data object over time; a remote performance storage pool for storing a copy of said data, available for immediate use as a backup copy of said data to provide business continuity to said data; a remote deduplicating storage pool for storing deduplicated data; and a controller for synchronizing the remote performance storage pool to have the second version of the data object using deduplicated data.

Abstract: Systems and methods for checking data integrity of a data object copied between storage pools in a storage system by comparing data samples copied from data objects. A series of successive copy operations are scheduled over time for copying a data object from a source data store to a target data store. A first data sample is generated based on a sampling scheme comprising an offset and a period. A second data sample is generated using a similar sampling scheme. The blocks of data in the first data sample and the second data sample are compared to determine if they differ to thereby indicate that the data object at the target store differs from the corresponding data object at the source data store.