Abstract: A computer-implemented method for providing increased scalability in deduplication storage systems may include (1) identifying a database that stores a plurality of reference objects, (2) determining that at least one size-related characteristic of the database has reached a predetermined threshold, (3) partitioning the database into a plurality of sub-databases capable of being updated independent of one another, (4) identifying a request to perform an update operation that updates one or more reference objects stored within at least one sub-database, and then (5) performing the update operation on less than all of the sub-databases to avoid processing costs associated with performing the update operation on all of the sub-databases. Various other systems, methods, and computer-readable media are also disclosed.

Abstract: Techniques for improving performance of a backup system are disclosed. In one particular exemplary embodiment, the techniques may be realized as a method for improving performance of a backup system. The method may comprise performing a backup of a client device, tracking, using at least one computer processor, references to data segments that are located outside of a unit of storage associated with the backup, calculating utilization of the unit of storage associated with the backup based on the tracked references, determining if the calculated utilization meets a specified parameter, and determining one or more responsive actions in the event the calculated utilization meets the specified parameter.

Abstract: A system and method for caching fingerprints in a client cache is provided. A data object that comprises a set of data segments and describes a backup process is identified. Thereafter, a request referencing the data object is made to a deduplication server to request that a task identifier be added to the data object. If the deduplication server is able to successfully add the task identifier to the data object, then an active identifier is added to each data segment from the set of data segments in a cache that is within a client system.

Abstract: A computer-implemented method for providing increased scalability in deduplication storage systems may include (1) identifying a database that stores a plurality of reference objects, (2) determining that at least one size-related characteristic of the database has reached a predetermined threshold, (3) partitioning the database into a plurality of sub-databases capable of being updated independent of one another, (4) identifying a request to perform an update operation that updates one or more reference objects stored within at least one sub-database, and then (5) performing the update operation on less than all of the sub-databases to avoid processing costs associated with performing the update operation on all of the sub-databases. Various other systems, methods, and computer-readable media are also disclosed.

Abstract: A system and method for backing up files to a single-instance storage system are disclosed. The files may be split into segments, and the file data may be stored in the single-instance storage system as individual segments. The single-instance storage system uses the concept of a file region which covers multiple segments of the file. If a region of a file is unchanged from one backup to the next, the system may use a region object to refer to the unchanged region. This avoids the need to update the reference information for each of the segments within the region, thus increasing the efficiency of backing up the new version of the file.

Abstract: A method and system for improving performance with single-instance-storage volumes by leveraging data locality is provided. A client provides a set of fingerprints generated from data segments to be saved to a single-instance storage volume and receives the information on whether a data segment exists on the single-instance storage volume and where it is stored if a data segment exists. Based on its received information, the client determines if a number of non-sequential accesses of a computer-readable medium for the said set of segments from the single-instance-storage volume exceeds a predetermined threshold. If so, the client provides the whole set of data segments for storage within the single-instance storage volume regardless of whether or not the data segments are duplicate data segments. These sent data segments will be stored contiguously within the single-instance storage volume while the duplicates will be removed from their previous stored locations.

Abstract: A system and method for caching fingerprints in a client cache is provided. A data object that comprises a set of data segments and describes a backup process is identified. Thereafter, a request referencing the data object is made to a deduplication server to request that a task identifier be added to the data object. If the deduplication server is able to successfully add the task identifier to the data object, then an active identifier is added to each data segment from the set of data segments in a cache that is within a client system.

Abstract: Providing authentication of users accessing an NFS shared file system. A shared secret is used as a component of the mount point used to access the NFS share. Upon receiving a request to access to the data in the NFS share, the process creates at least one cryptographic value and then creates a path to the mount point which incorporates the cryptographic value. The process then creates the mount point at the path, e.g., /PATH:k1, where k1 is the cryptographic value. Creation of the mount point is preferably performed using NFS protocol semantics, without requiring any changes to the NFS protocol semantics. A second cryptographic value, k2, may be used for unmounting the mount point.

Abstract: Containers that store data objects that were written to those containers during a particular backup are accessed. Then, a subset of the containers is identified; the containers in the subset have less than a threshold number of data objects associated with the particular backup. Data objects that are in containers in that subset and that are associated with the backup are copied to one or more other containers. Those other containers are subsequently used to restore data objects associated with the backup.

Abstract: The present disclosure provides for efficiently creating a full backup image of a client device by efficiently communicating backup data to a backup server using a change tracking log, or track log. A present full backup image can be created using a track log that is associated with a previous full backup image. The client device can determine whether files, which were included in the previous full backup image, have or have not changed using the track log. The client device can transmit changed file data to the backup server for inclusion in the present full backup image. The client device can also transmit metadata identifying unchanged file data to the backup server. The backup server can use the metadata to extract a copy of the unchanged file data from the previous full backup image for inclusion in the present full backup image.

Abstract: A computer-implemented method for managing data in cloud storage using deduplication techniques is described. At least one unique segment of data is received from at least one backup source. The at least one unique segment of data is written to at least one container file associated with the at least one back up source. An index file is created for the at least one container file. The index file includes information regarding the unique segments of data. A plurality of the unique segments of data is transmitted from the at least one container file as an object to a virtual container on cloud storage. A virtual container is composed of an index file and one or more container files, and the index file and each container file are stored as individual cloud storage objects. Spatial locality of the plurality of unique segments of data is maintained in cloud storage.

Abstract: A method and system for efficient space management for single-instance-storage volumes is provided. A backup module storing data within a collection of containers according to access locality and retention time of the data, wherein the retention time defines an amount of time the data is stored within the collection of containers before deletion of the data, and the access locality comprises an order in which the data is to be accessed is further provided. A compaction module compacting the stored data by selecting at least two containers among the collection of containers, wherein the selection is performed using a predetermined criteria that includes access locality and retention time of the data is also provided. The compaction module distributes the data among the at least two containers. The compaction criteria creates an imbalance among the containers to create more empty, full, or nearly full containers.

Abstract: A computer-implemented method for migrating an object from a deduplication store to an external domain in an external media is described. A deduplication store (dstore) deduplication map (dmap) is retrieved from a dstore for an object stored in the dstore. A determination is made as to whether an external dmap exists in the external domain for an object referenced in the dstore dmap. If the external dmap exists, a determination is made as to whether data referenced in the dstore dmap exist in the external dmap. If the referenced data do not exist, the referenced data is extracted from the dstore to the external domain. If the external dmap does not exist, the object and the associated dmap are extracted from the dstore to the external domain.

Abstract: A method, system and apparatus for efficient storage of small files in a segment-based deduplication scheme by allocating multiple small files to a single data segment is provided. A mechanism for distinguishing between large files (e.g., files that are on the order of the size of a segment or larger) and smaller files, and starting a new segment at the beginning of a large file is also provided. A file attribute-based system for determining an identity of a small file at which to begin a new segment and then allocating subsequent small files to that segment and contiguous segments until a next small file having an appropriate attribute subsequently is encountered to begin a new segment is further provided. In one aspect of the present invention a filename hash is used for file attribute analysis to determine when a new segment should begin.

Abstract: A computer-implemented method for deduplicating data is disclosed. The method may include identifying a request to transfer data from a client in a deduplication system to a server in the deduplication system. The method may also include determining, based at least in part on performance of the deduplication system, whether to deduplicate the data before transferring the data from the client to the server. The method may further include performing the following, the order of which is based on the determination: deduplicating the data and transferring the data from the client to the server. Various other methods, systems, and computer-readable media are also disclosed.

Abstract: Various embodiments of a computer system and methods are disclosed. In one embodiment, a computer system includes hosts coupled to a backup medium. The backup medium stores data segments and objects, each object referencing one or more segments. A second host conveys a write request to a first host identifying a data entity comprising segments referenced by a first object. The first host restores a copy of the data entity and creates a second object referencing the segments that were referenced by the first object in response to receiving the write request. The first host adds segment references to the second object in response to receiving a change that adds data to the copy or deletes a portion of a segment from the copy. The first host removes a segment reference from the second object if a change deletes all of a segment from the copy.

Abstract: Containers that store data objects that were written to those containers during a particular backup are accessed. Then, a subset of the containers is identified; the containers in the subset have less than a threshold number of data objects associated with the particular backup. Data objects that are in containers in that subset and that are associated with the backup are copied to one or more other containers. Those other containers are subsequently used to restore data objects associated with the backup.

Abstract: A system, method, and medium for dynamically scaling the size of a fingerprint index in a deduplication storage system. Fingerprints are stored as entries in a fingerprint index, and the fingerprint index is scaled to fit into an in-memory cache to enable fast accesses to the index. A persistent copy of the full fingerprint index is stored on a non-volatile memory. The cached fingerprint index uses binary sampling to categorize half of the fingerprint entries as samples and protected, and the other half of the entries as non-samples and replaceable. When a search of the cached index results in a hit on a sample entry, all of the non-sample entries associated with the same container are copied from the persistent index to the cached index.

Abstract: Data-object-level replication is performed based on storage-device-level change tracking. A method of replicating a data object (such as a file) involves accessing mapping information and change information. The mapping information identifies that one or more regions of a storage device (such as a logical volume) store portions of the data object. The change information identifies whether the regions of the storage device, which were identified by the mapping information, have been modified subsequent to a particular point in time. If the change information identifies that any of those regions of the storage device have been modified, replication of each modified one of the regions to a secondary system is initiated. Replication can be initiated from a system other than the production system.

Abstract: A method and system for efficient space management for single-instance-storage volumes is provided. A backup module storing data within a collection of containers according to access locality and retention time of the data, wherein the retention time defines an amount of time the data is stored within the collection of containers before deletion of the data, and the access locality comprises an order in which the data is to be accessed is further provided. A compaction module compacting the stored data by selecting at least two containers among the collection of containers, wherein the selection is performed using a predetermined criteria that includes access locality and retention time of the data is also provided. The compaction module distributes the data among the at least two containers. The compaction criteria creates an imbalance among the containers to create more empty, full, or nearly full containers.