Abstract: Methods, computer systems, and computer program products for processing data a computing environment are provided. The computer environment for data deduplication storage receives a plurality of write operations for deduplication storage of the data. The data is buffered in a plurality of buffers with overflow temporarily stored to a memory hierarchy when the data received for deduplication storage is sequential or non sequential. The data is accumulated and updated in the plurality of buffers per a data structure, the data structure serving as a fragment map between the plurality of buffers and a plurality of user file locations. The data is restructured in the plurality of buffers to form a complete sequence of a required sequence size. The data is provided as at least one stream to a stream-based deduplication algorithm for processing and storage.

Abstract: In a hash-based multi-tenancy in a deduplication system, incorporating, as if part of input data, a tenant identification (ID) into a hash value calculation using a single hash based index table for separating data segments in a multi-tenant deduplication system.

Abstract: A snapshot of selected objects in a source repository is created in response to the user-initiated replication. The snapshot is designated as a snapshot replication job. Unsynchronized objects in a target destination are detected by comparing a state of the selected objects in the snapshot with a current state of the target destination at the time of execution of the snapshot replication job. An object identification (ID) and a state ID are sent to the target destination and used to locate and compare the selected object with a current state of the selected object in the target destination.

Abstract: For efficient high availability for a multi-node cluster using a processor device in a computing environment, using duplicate, standby host-bus adaptors (HBAs) for alternate nodes with respect to a node with the duplicate, standby HBAs using duplicate credentials of active HBAs of the node for shutting down the node, taking an active HBA of the node offline, and/or activating one of the alternate nodes. A failed node is shut down by an active one of the plurality of alternate nodes using a fence device, and all jobs of the failed node are taken over by the active node using a shared data structure including SCSI device assignments, SCSI reservations, and required host notifications.

Abstract: For efficient reclamation of pre-allocated direct memory access (DMA) memory in a computing environment, hot-add random access memory (RAM) is emulated for a general purpose use by reclamation of pre-allocated DMA memory reserved at boot time by notifying a non-kernel use device user that the non-kernel use device has a smaller window, stopping and remapping to the smaller window, and notifying a kernel that new memory has been added, wherein the new memory is a region left after the remap. The hot-add RAM is split into at least two continuous parts.

Abstract: For producing digest block segmentations based on reference segmentations in a data deduplication system using a processor device in a computing environment, digests are calculated for an input data chunk. Data matches and data mismatches are produced based on matching input digests with reference digests. Secondary digest block segmentations are obtained from similar reference intervals for each of the data mismatches and applied to the input data.

Abstract: Methods, computer systems, and computer program products for processing data a computing environment are provided. The computer environment for data deduplication storage receives a plurality of write operations for deduplication storage of the data. The data is buffered in a plurality of buffers with overflow temporarily stored to a memory hierarchy when the data received for deduplication storage is sequential or non sequential. The data is accumulated and updated in the plurality of buffers per a data structure, the data structure serving as a fragment map between the plurality of buffers and a plurality of user file locations. The data is restructured in the plurality of buffers to form a complete sequence of a required sequence size. The data is provided as at least one stream to a stream-based deduplication algorithm for processing and storage.

Abstract: Input data is partitioned into data chunks and digest values are calculated for each of the data chunks. The positions of similar repository data are found in a repository of data for each of the data chunks. The input digests of the input data are matched with the repository digests contained in the global digests cache for locating data matches. The processor prefers to match the input digests of the input data with the repository digests contained in the global digests cache which are of the similar repository data, rather than repository digests which are of other repository data that was not determined as similar to the input data chunks. The positions of the similar repository data are used to locate and linearly load into the global digests cache, digests and digest block boundaries of the similar repository data.

Abstract: For utilizing a global digests cache in deduplication processing in a data deduplication system using a processor device in a computing environment, input data is partitioned into data chunks and digest values are calculated for each of the data chunks. The positions of similar repository data are found in a repository of data for each of the data chunks. The input digests of the input data are matched with the repository digests contained in the global digests cache for locating data matches. The positions of the similar repository data are used to locate and linearly load into the global digests cache, digests and digest block boundaries of the similar repository data in a sequence corresponding to a placement order of calculated values of the digests of the similar repository data.

Abstract: Input data is partitioned into data chunks and digest values are calculated for each of the data chunks. The positions of similar repository data are found in a repository of data for each of the data chunks. The input digests of the input data are matched with the repository digests contained in the global digests cache for locating data matches. A sample of the repository digests is loaded into a search mechanism within the global digests cache. The positions of the similar repository data are used to locate and linearly load into the global digests cache, digests and digest block boundaries of the similar repository data in a sequence corresponding to a placement order of calculated values of the digests of the similar repository data.

Abstract: Exemplary embodiments for optimizing disaster recovery systems during takeover operations are provided. In one embodiment, by way of example only, a flag is set in a replication grid manager to identify replication grid members to consult in a reconciliation process for resolving intersecting and non-intersecting data amongst the disaster recovery systems for a takeover operation, including indicating those of the replication grid members that acquired ownership over cartridges belonging to source systems.

Abstract: Various embodiments for managing data in a data storage having data deduplication. A back reference data structure is configured for user data segments as a mechanism to identify an affected storage block to which information in the back reference data structure refers. The back reference data structure is initialized such that a resolution of the back reference data structure diminishes as a number of the user data segments referencing the affected storage block increases.

Abstract: Methods, computer systems, and computer program products for processing data a computing environment are provided. The computer environment for data deduplication storage receives a plurality of write operations for deduplication storage of the data. The data is buffered in a plurality of buffers with overflow temporarily stored to a memory hierarchy when the data received for deduplication storage is sequential or non sequential. The data is accumulated and updated in the plurality of buffers per a data structure, the data structure serving as a fragment map between the plurality of buffers and a plurality of user file locations. The data is restructured in the plurality of buffers to form a complete sequence of a required sequence size. The data is provided as at least one stream to a stream-based deduplication algorithm for processing and storage.

Abstract: Input data is partitioned into data chunks and digest values are calculated for each of the data chunks. The positions of similar repository data are found in a repository of data for each of the data chunks. The input digests of the input data are matched with the repository digests contained in the global digests cache for locating data matches. The processor prefers to match the input digests of the input data with the repository digests contained in the global digests cache which are of the similar repository data, rather than repository digests which are of other repository data that was not determined as similar to the input data chunks. The positions of the similar repository data are used to locate and linearly load into the global digests cache, digests and digest block boundaries of the similar repository data.

Abstract: For utilizing a global digests cache in deduplication processing in a data deduplication system using a processor device in a computing environment, input data is partitioned into data chunks and digest values are calculated for each of the data chunks. The positions of similar repository data are found in a repository of data for each of the data chunks. The input digests of the input data are matched with the repository digests contained in the global digests cache for locating data matches. The positions of the similar repository data are used to locate and linearly load into the global digests cache, digests and digest block boundaries of the similar repository data in a sequence corresponding to a placement order of calculated values of the digests of the similar repository data.

Abstract: For self-healing in a hash-based deduplication system using a processor device in a computing environment, deduplication digests of data and a corresponding list of the deduplication digests in a table of contents (TOC) are maintained for the self-healing of data that is lost or unreadable. The input data digests are compared to the TOC if directed to data that is lost or unreadable, and the input data digests are used to repair the one of lost and unreadable data.

Abstract: A snapshot of selected objects in a source repository is created in response to the user-initiated replication. The snapshot is designated as a snapshot replication job. Unsynchronized objects in a target destination are detected by comparing a state of the selected objects in the snapshot with a current state of the target destination at the time of execution of the snapshot replication job. An object identification (ID) and a state ID are sent to the target destination and used to locate and compare the selected object with a current state of the selected object in the target destination.

Abstract: Input data is partitioned into data chunks and digest values are calculated for each of the data chunks. The positions of similar repository data are found in a repository of data for each of the data chunks. The input digests of the input data are matched with the repository digests contained in the global digests cache for locating data matches. A sample of the repository digests is loaded into a search mechanism within the global digests cache. The positions of the similar repository data are used to locate and linearly load into the global digests cache, digests and digest block boundaries of the similar repository data in a sequence corresponding to a placement order of calculated values of the digests of the similar repository data.

Abstract: For efficient high availability for a multi-node cluster using a processor device in a computing environment, using duplicate, standby host-bus adaptors (HBAs) for alternate nodes with respect to a node with the duplicate, standby HBAs using duplicate credentials of active HBAs of the node for shutting down the node, taking an active HBA of the node offline, and/or activating one of the alternate nodes. A failed node is shut down by an active one of the plurality of alternate nodes using a fence device, and all jobs of the failed node are taken over by the active node using a shared data structure including SCSI device assignments, SCSI reservations, and required host notifications.

Abstract: Input data is partitioned into data chunks and digest values are calculated for each of the data chunks. The positions of similar repository data are found in a repository of data for each of the data chunks. The repository digests of the similar repository data are located and loaded into the global digests cache. The global digests cache contains digests previously loaded by other deduplication processes. The input digests of the input data are matched with the repository digests contained in the global digests cache for locating data matches. A sample of the repository digests is loaded into a search mechanism within the global digests cache.