Abstract: A method includes creating a deduplicated universal share (US) of data objects, which in turn includes receiving a US of the data objects, deduplicating the US, wherein deduplicating the US includes: hashing segments of the US to generate respective US segment fingerprints; comparing US segment fingerprints to fingerprints for respective segments held in deduplication storage in order to identify segments in the deduplication storage that equate to the US segments, respectively, of the US; storing identifiers that directly or indirectly identify locations, respectively, of the segments, respectively, in the deduplication storage that equate to the US segments, respectively, of the US. After creating the deduplicated universal share, a deduplicated backup of the US is created without reassembling the US from segments held in the deduplication storage, the creating the deduplicated backup including: creating a list that comprises copies of the stored identifiers, and storing the list.

Abstract: Methods, computer program products, computer systems, and the like for efficient metadata management are disclosed, which can include determining whether a change in a status of data has occurred. In response to a determination that the change has occurred, such methods, computer program products, and computer systems can include determining whether a move condition has been met, and, in response to a determination that the move condition has been met, moving the metadata from the first storage unit to a second storage unit.

Abstract: Methods, computer program products, computer systems, and the like are disclosed that provide for scalable deduplication. Such methods, computer program products, and computer systems can include, in response to receiving a request to perform a lookup operation, performing the lookup operation and, in response to the signature not being found, forwarding the request to a remote node. Further, in response to receiving an indication that the signature was not found at the remote node, processing the subunit of data as a unique subunit of data.

Abstract: Methods, computer program products, computer systems, and the like are disclosed that provide for scalable deduplication in an efficient and effective manner. For example, such methods, computer program products, and computer systems can include receiving a data object at an assigned node, determining whether the data object includes a sub-data object, and processing the sub-data object. The assigned node is a node of a plurality of nodes of a cluster, where the data object includes a data segment, and a signature. The signature is generated based, at least in part, on data of the data segment. The processing includes sending the sub-data object to a remote node. The remote node is another node of the plurality of nodes of the cluster.

Abstract: Methods, computer program products, and computer systems for the management of data references in an efficient and effective manner are disclosed. Such methods, computer program products, and computer systems include receiving a change tracking stream at the computer system, identifying a data object group, and performing a deduplication management operation on the data object group. The change tracking stream is received from a client computing system. The change tracking stream identifies one or more changes made to a plurality of data objects of the client computing system. The identifying is based, at least in part, on at least a portion of the change tracking stream. The data object group represents the plurality of data objects.

Abstract: Methods, computer program products, and computer systems for the management of data references in an efficient and effective manner are disclosed. Such methods, computer program products, and computer systems include receiving a change tracking stream at the computer system, identifying a data object group, and performing a deduplication management operation on the data object group. The change tracking stream is received from a client computing system. The change tracking stream identifies one or more changes made to a plurality of data objects of the client computing system. The identifying is based, at least in part, on at least a portion of the change tracking stream. The data object group represents the plurality of data objects.

Abstract: Methods, computer program products, computer systems, and the like for efficient metadata management are disclosed, which can include receiving a subunit of storage, storing a first metadata portion of the subunit of storage in a first unit of storage, and storing a second metadata portion of the subunit of storage in a second unit of storage.

Abstract: Disclosed herein are systems, methods, and processes to perform replication between heterogeneous storage systems. In one embodiment, a request to perform a replication operation is sent by a target server to a source server, where the target server and the source server use different protocols to store data. A plurality of instructions, which are associated with a replication stream, are received from the source server by the target server, where the plurality of instructions comprise an include instruction to include existing data and a write instruction to write new data. A replication stream, which is associated with a backup stream stored on the source server, is also received from the source server, where the replication stream and the backup stream share a common format. The target server synthesizes a new replicated backup image, where the synthesizing comprises performing the include instruction and the write instruction on the replication stream.

Abstract: A method and apparatus is disclosed for optimized backups. In one embodiment, the method includes creating a deduplicated universal share of data objects, which in turn includes receiving a universal share of the data objects, deduplicating the universal share, wherein deduplicating the universal share includes: hashing segments of the universal share to generate respective universal share segment fingerprints; comparing the universal share segment fingerprints to fingerprints for respective segments held in deduplication storage in order to identify segments in the deduplication storage that equate to the universal share segments, respectively, of the universal share; storing identifiers that directly or indirectly identify locations, respectively, of the segments, respectively, in the deduplication storage that equate to the universal share segments, respectively, of the universal share.

Abstract: The disclosed computer-implemented method for data placement in container-based storage systems may include (i) identifying a file stored within a container-based storage system, where the container-based storage system stores the file as data segments within containers, (ii) receiving, in response to a write operation directed to the file, a request to store within the container-based storage system a new data segment generated by the write operation, (iii) describing the file in terms of a plurality of consecutive slabs, (iv) determining that the new data segment falls within a specified slab, and (v) fulfilling the request to store the new data segment within the container-based storage system by storing the new data segment in a designated container that corresponds to the specified slab in response to determining that the new data segment falls within the specified slab. Various other methods, systems, and computer-readable media are also disclosed.

Abstract: Methods, computer program products, computer systems, and the like are disclosed that provide for scalable deduplication in an efficient and effective manner. For example, such methods, computer program products, and computer systems can include receiving a data object at an assigned node, determining whether the data object includes a sub-data object, and processing the sub-data object. The assigned node is a node of a plurality of nodes of a cluster, where the data object includes a data segment, and a signature. The signature is generated based, at least in part, on data of the data segment. The processing includes sending the sub-data object to a remote node. The remote node is another node of the plurality of nodes of the cluster.

Abstract: Disclosed are methods, systems, and processes to improve extent map performance A request for a data block is received. In response to detecting a cache miss, a temporary table is searched for the data block. If the data block is not found in the temporary table, a base table is searched for the data block.

Abstract: Methods, computer program products, and computer systems for the management of data references in an efficient and effective manner are disclosed. Such methods, computer program products, and computer systems include receiving a change tracking stream at the computer system, identifying a data object group, and performing a deduplication management operation on the data object group. The change tracking stream is received from a client computing system. The change tracking stream identifies one or more changes made to a plurality of data objects of the client computing system. The identifying is based, at least in part, on at least a portion of the change tracking stream. The data object group represents the plurality of data objects.

Abstract: Disclosed herein are systems, methods, and processes to perform replication between heterogeneous storage systems. In one embodiment, a request to perform a replication operation is sent by a target server to a source server, where the target server and the source server use different protocols to store data. A plurality of instructions, which are associated with a replication stream, are received from the source server by the target server, where the plurality of instructions comprise an include instruction to include existing data and a write instruction to write new data. A replication stream, which is associated with a backup stream stored on the source server, is also received from the source server, where the replication stream and the backup stream share a common format. The target server synthesizes a new replicated backup image, where the synthesizing comprises performing the include instruction and the write instruction on the replication stream.

Abstract: The present disclosure describes storing backup images in backup storage using snapshots. In response to receipt of a backup image that includes a complete copy of client data, a storage unit is generated and the backup image is stored in the storage unit. In response to receipt of a streamlined backup image that includes changed client data and metadata identifying unchanged client data in a base backup image, a base storage unit that includes the base backup image is identified, a snapshot is taken of the base storage unit (which generates a snapshot), and the streamlined backup image is stored on the snapshot. The metadata of the streamlined backup image is analyzed to identify portions of the base backup image that are not referenced by the streamlined backup image, which are removed from the snapshot.

Abstract: Disclosed herein are methods, systems, and processes to improve backup performance after backup failure. It is determined whether a backup operation is successful. In response to a determination that the backup operation was unsuccessful, backup metadata associated with the backup operation is replaced with snapshot metadata associated with a snapshot that is taken at a first point in time that is prior to initiation of the backup operation. A rolled-back snapshot that permits another backup operation to be performed at a second point in time based on the snapshot metadata is then generated.

Abstract: Disclosed herein are systems, methods, and processes to perform replication between heterogeneous storage systems. Information associated with a backup stream is recorded during a backup operation by a source server and includes instructions. The instructions include an include instruction to include existing data and a write instruction to write new data during a replication operation. A request to perform the replication operation is received. In response to the request, the information is sent to a target server as part of performing the replication operation.

Abstract: A method for grouping backup images in deduplication groups is described. In one embodiment, the method includes identifying a new backup image, obtaining metadata from one or more nodes in the new backup image, and comparing the metadata from the one or more nodes in the new backup image with information from one or more nodes in a backup image from a first deduplication group. Each of the one or more nodes include at least in part a file or a directory. The first deduplication group is one group from a plurality of deduplication groups.

Abstract: A computer-implemented method for increasing restore speeds of backups stored in deduplicated storage systems may include (1) identifying a backup that includes data stored in at least one data container within a deduplicated storage system, (2) detecting a subsequent backup that includes additional data, (3) calculating an amount of duplication between the additional data included in the subsequent backup and the data stored in the data container, (4) determining that the amount of duplication between the additional data and the data stored in the data container is below a predetermined threshold, (5) identifying at least one additional data container to store the additional data instead of deduplicating the additional data with respect to the data container, and then (6) storing the additional data in the additional data container to facilitate increasing a restore speed of the subsequent backup. Various other methods, systems, and computer-readable media are also disclosed.

Abstract: The disclosed computer-implemented method for reducing data fragmentation may include (1) identifying update data which updates an initial data set; (2) categorizing, using a database manager, the update data based on how the update data is expected to impact digital storage device resources; (3) storing the update data in a physical storage device in a physical order based on the category of the update data; and (4) updating, after storing the update data, an extent map indicating a mapping of logical locations of the update data to physical locations at which the update data is stored. Various other methods, systems, and computer-readable media are also disclosed.