Abstract: Input data is partitioned into data chunks and digest values are calculated for each of the data chunks. The positions of similar repository data are found in a repository of data for each of the data chunks. The repository digests of the similar repository data are located and loaded into the global digests cache. The global digests cache contains digests previously loaded by other deduplication processes. The input digests of the input data are matched with the repository digests contained in the global digests cache for locating data matches. The processor prefers to match the input digests of the input data with the repository digests contained in the global digests cache which are of the similar repository data, rather than repository digests which are of other repository data that was not determined as similar to the input data chunks.

Abstract: A snapshot of selected objects in a source repository is created in response to the user-initiated replication. The snapshot is designated as a snapshot replication job. Unsynchronized objects in a target destination are detected by comparing a state of the selected objects in the snapshot with a current state of the target destination at the time of execution of the snapshot replication job. One of the selected objects is retrieved in an ordered manner from the snapshot. An object identification (ID) and a state ID are sent to the target destination and used to locate and compare the selected object with a current state of the selected object in the target destination.

Abstract: For utilizing a global digests cache in deduplication processing in a data deduplication system using a processor device in a computing environment, input data is partitioned into data chunks and digest values are calculated for each of the data chunks. The positions of similar repository data are found in a repository of data for each of the data chunks. The repository digests of the similar repository data are located and loaded into the global digests cache. The global digests cache contains digests previously loaded by other deduplication processes. The input digests of the input data are matched with the repository digests contained in the global digests cache for locating data matches.

Abstract: For efficient high availability for a multi-node cluster using a processor device in a computing environment, using duplicate, standby host-bus adaptors (HBAs) for alternate nodes with respect to a node with the duplicate, standby HBAs using duplicate credentials of active HBAs of the node for shutting down the node, taking an active HBA of the node offline, and/or activating one of the alternate nodes.

Abstract: Methods, computer systems, and computer program products for processing data a computing environment are provided. The computer environment for data deduplication storage receives a plurality of write operations for deduplication storage of the data. The data is buffered in a plurality of buffers with overflow temporarily stored to a memory hierarchy when the data received for deduplication storage is sequential or non sequential. The data is accumulated and updated in the plurality of buffers per a data structure, the data structure serving as a fragment map between the plurality of buffers and a plurality of user file locations. The data is restructured in the plurality of buffers to form a complete sequence of a required sequence size. The data is provided as at least one stream to a stream-based deduplication algorithm for processing and storage.

Abstract: Data file in the data deduplication system are associated with a file identifier defined to have a first part identifier for denoting a location of the data file in a storage, and a second part identifier for uniquely identifying the data file in the data deduplication system over time.

Abstract: Deduplicated data is packed in a self-contained deduplicated repository having unique data blocks with each being referenced by a globally unique identifier (GUID). The self-contained deduplicated repository has information regarding both deduplicated data files and the unique data blocks of each of the deduplicated data files and a master GUID list containing a location of each of the unique data blocks.

Abstract: Expired files in the deduplicating virtual media are selectively erased using a backup application for notifying a backup repository of which expired files are no longer required. The space of the expired files is reclaimed for reuse. Virtual space of the expired files is reserved for allowing the backup application to seek past the reclaimed space to subsequent data in the deduplicating virtual media. Expired files are identified and positions of the expired files are determined in the deduplicating virtual media thereby converting the positions into a list of expired files that is sent to the backup repository.

Abstract: Deduplicated data is packed in a self-contained deduplicated repository having unique data blocks with each being referenced by a globally unique identifier (GUID). The self-contained deduplicated repository has information regarding both deduplicated data files and the unique data blocks of each of the deduplicated data files and a master GUID list containing a location of each of the unique data blocks.

Abstract: For efficient reclamation of pre-allocated direct memory access (DMA) memory in a computing environment, hot-add random access memory (RAM) is emulated for a general purpose use by reclamation of pre-allocated DMA memory reserved at boot time by notifying a non-kernel use device user that the non-kernel use device has a smaller window, stopping and remapping to the smaller window, and notifying a kernel that new memory has been added, wherein the new memory is a region left after the remap. The hot-add RAM is split into at least two continuous parts.

Abstract: A plurality of server processor workers is configured for the distributed parallel processing of deduplicated data entities in a plurality of chunks. The deduplicated data processing rate is regulated using a rate control mechanism. The rate control mechanism incorporates a debt/credit algorithm specifying which of the plurality of workers processing the deduplicated data entities must wait for each of a plurality of calculated required sleep times, the calculated required sleep times being calculated as a best fit between a maximum allowable sleep time, a delta vector, and a limit per time vector, wherein the required sleep time is one of the plurality of calculated required sleep times.

Abstract: Exemplary method, system, and computer program product embodiments for protecting data segments by a processor device in a computing environment, are provided. In one embodiment, by way of example only, a history table is configured to accompany data segments for consultation during a replication operation. If the history table indicates an ownership conflict, the replication operation is inhibited.

Abstract: For efficient calculation of both similarity search values and boundaries of digest blocks in data deduplication, input data is partitioned into chunks, and for each chunk a set of rolling hash values is calculated. A single linear scan of the rolling hash values is used to produce both similarity search values and boundaries of the digest blocks of the chunk. The rolling hash values are used to contribute to the calculation of the similarity search values and to the calculation of the boundaries of the digest blocks.

Abstract: Various embodiments for protecting data segments by a processor device in a computing environment, are provided. In one embodiment, a method comprises, if an error occurs during a replication operation from a source location to a destination location causing at least one of a plurality of data segments to be deleted in the source location, performing: executing a failback operation by a reverse replication process from the destination location to the source location, and regaining ownership of the data segments by the source location; and pursuant to the failback operation, performing: enabling the reverse replication process if an owner of the data segments is a designated one of the destination location, and adding an invalid owner notification in a history table, wherein the failback operation is identified for the designated one of the at least one destination location.

Abstract: For efficient reclamation of pre-allocated direct memory access (DMA) memory in a computing environment, hot-add random access memory (RAM) is emulated for a general purpose use by reclamation of pre-allocated DMA memory reserved at boot time for responding to an emergency by notifying a non-kernel use device user that the non-kernel use device has a smaller window, stopping and remapping to the smaller window, and notifying a kernel that new memory has been added, wherein the new memory is a region left after the remap.

Abstract: For digest retrieval based on similarity search in deduplication processing in a data deduplication system using a processor device in a computing environment, input data is partitioned into fixed sized data chunks. Similarity elements and digest block boundaries and digest values are calculated for each of the fixed sized data chunks. Matching similarity elements are searched for in a search structure containing the similarity elements for each of the fixed sized data chunks in a repository of data. Positions of similar data are located in the repository. The positions of the similar data are used to locate and load into the memory stored digest values and corresponding stored digest block boundaries of the similar data in the repository. The digest values and the corresponding digest block boundaries of the input data are matched with the stored digest values and the corresponding stored digest block boundaries to find data matches.

Abstract: For adding cooperative file coloring in a similarity based deduplication system using a processor device in a computing environment. Input streams of backup application are parsed and the data segments are marked with file coloring that represents a source file of the backup application for comparing the data segments to determine if the data segments are identical during a similarity check operation.

Abstract: Dynamically creates a cloned target volume by cloning a production volume without breaking a Flashcopy chain for continuing an ingestion of a Flashcopy backup target volume.

Abstract: Various embodiments for managing data in a data storage having data deduplication. A back reference data structure is configured for user data segments as a mechanism to identify an affected storage block to which information in the back reference data structure refers. The back reference data structure is initialized such that a resolution of the back reference data structure diminishes as a number of the user data segments referencing the affected storage block increases.

Abstract: For write cache protection of purpose built backup appliances in a computing environment, backup data of the write cache is created using a server memory that includes the write cache in a redundant array of independent disks (RAID) system. The server memory is not controlled by a failing RAID card and/or NVRAM card that controls the write cache. The backup data in the server memory is flushed to a persistent storage device subsequent to detecting a failure of the write cache for enabling recovery of the backup data using the persistent storage device during a reboot operation.