Abstract: Methods, computer systems, and computer program products for processing data a computing environment are provided. The computer environment for data deduplication storage receives a plurality of write operations for deduplication storage of the data. The data is buffered in a plurality of buffers with overflow temporarily stored to a memory hierarchy when the data received for deduplication storage is sequential or non sequential. The data is accumulated and updated in the plurality of buffers per a data structure, the data structure serving as a fragment map between the plurality of buffers and a plurality of user file locations. The data is restructured in the plurality of buffers to form a complete sequence of a required sequence size. The data is provided as at least one stream to a stream-based deduplication algorithm for processing and storage.

Abstract: Methods, computer systems, and computer program products for processing data a computing environment are provided. The computer environment for data deduplication storage receives a plurality of write operations for deduplication storage of the data. The data is buffered in a plurality of buffers with overflow temporarily stored to a memory hierarchy when the data received for deduplication storage is sequential or non sequential. The data is accumulated and updated in the plurality of buffers per a data structure, the data structure serving as a fragment map between the plurality of buffers and a plurality of user file locations. The data is restructured in the plurality of buffers to form a complete sequence of a required sequence size. The data is provided as at least one stream to a stream-based deduplication algorithm for processing and storage.

Abstract: Methods, computer systems, and computer program products for processing data a computing environment are provided. The computer environment for data deduplication storage receives a plurality of write operations for deduplication storage of the data. The data is buffered in a plurality of buffers with overflow temporarily stored to a memory hierarchy when the data received for deduplication storage is sequential or non sequential. The data is accumulated and updated in the plurality of buffers per a data structure, the data structure serving as a fragment map between the plurality of buffers and a plurality of user file locations. The data is restructured in the plurality of buffers to form a complete sequence of a required sequence size. The data is provided as at least one stream to a stream-based deduplication algorithm for processing and storage.

Abstract: Methods, computer systems, and computer program products for processing data a computing environment are provided. The computer environment for data deduplication storage receives a plurality of write operations for deduplication storage of the data. The data is buffered in a plurality of buffers with overflow temporarily stored to a memory hierarchy when the data received for deduplication storage is sequential or non sequential. The data is accumulated and updated in the plurality of buffers per a data structure, the data structure serving as a fragment map between the plurality of buffers and a plurality of user file locations. The data is restructured in the plurality of buffers to form a complete sequence of a required sequence size. The data is provided as at least one stream to a stream-based deduplication algorithm for processing and storage.

Abstract: Data file in the data deduplication system are associated with a file identifier defined to have a first part identifier for denoting a location of the data file in a storage, and a second part identifier for uniquely identifying the data file in the data deduplication system over time.

Abstract: For optimizing a partition of a data block into matching and non-matching segments in data deduplication using a processor device in a computing environment, a sequence of matching segments is split into sub-parts for obtaining a globally optimal subset, to which an optimal calculation is applied. The solutions of optimal calculations for the entire range of the sequence are combined, and a globally optimal subset is built by means of a first two-dimensional table represented by a matrix C[i,j], and storing a representation of the globally optimal subset in a second two-dimensional table represented by a matrix PS[i,j] that holds, at entry [i,j] of the matrix, the globally optimal subset for a plurality of parameters in form of a bit-string of length j?i+1, wherein i and j are indices of bit positions corresponding to segments.

Abstract: For optimizing a partition of a data block into matching and non-matching segments in data deduplication using a processor device in a computing environment, a sequence of matching segments is split into sub-parts for obtaining a globally optimal subset, to which an optimal calculation is applied. The solutions of optimal calculations for the entire range of the sequence are combined, and a globally optimal subset is built by means of a first two-dimensional table represented by a matrix C[i, j], and storing a representation of the globally optimal subset in a second two-dimensional table represented by a matrix PS[i, j] that holds, at entry [i, j] of the matrix, the globally optimal subset for a plurality of parameters in form of a bit-string of length j?i+1, wherein i and j are indices of bit positions corresponding to segments.

Abstract: For optimizing a partition of a data block into matching and non-matching segments in data deduplication using a processor device in a computing environment, an optimal calculation operation is applied in polynomial time to the matching segments for selecting a globally optimal subset of a set of matching segments according to overhead considerations for minimizing an overall size of a deduplicated file by determining a trade off between a time complexity and a space complexity.

Abstract: Data file in the data deduplication system are associated with a file identifier defined to have a first part identifier for denoting a location of the data file in a storage, and a second part identifier for uniquely identifying the data file in the data deduplication system over time.

Abstract: Various embodiments for replicating deduplicated data using a processor device are provided. A block of the deduplicated data, created in a source repository, is assigned a global block identifier (ID) unique in a grid set inclusive of the source repository. The global block ID is generated using at least one unique identification value of the block, a containing grid of the grid set, and the source repository. The global block ID is transmitted from the source repository to a target repository. If the target repository determines the global block ID is associated with an existing block of the deduplicated data located within the target repository, the block is not transmitted to the target repository during a subsequent replication process.

Abstract: Methods for managing resources for maintenance tasks in computing systems are provided. One system includes a controller and memory coupled to the controller, the memory configured to store a module. The controller, when executing the module, is configured to determine an amount of available resources for use by a plurality of maintenance tasks in a computing system and divide the available resources between the plurality of maintenance tasks based on a need for each maintenance task. One method includes determining, by a central controller, an amount of available resources for use by a plurality of maintenance tasks in a computing system and dividing the available resources between the plurality of maintenance tasks based on a need for each maintenance task. Computer storage mediums including a computer program product method for managing resources for maintenance tasks in computing systems are also provided.

Abstract: Systems for managing resources for maintenance tasks in computing systems are provided. One system includes a controller and memory coupled to the controller, the memory configured to store a module. The controller, when executing the module, is configured to determine an amount of available resources for use by a plurality of maintenance tasks in a computing system and divide the available resources between the plurality of maintenance tasks based on a need for each maintenance task. One method includes determining, by a central controller, an amount of available resources for use by a plurality of maintenance tasks in a computing system and dividing the available resources between the plurality of maintenance tasks based on a need for each maintenance task. Computer storage mediums including a computer program product method for managing resources for maintenance tasks in computing systems are also provided.

Abstract: For optimizing a partition of a data block into matching and non-matching segments in data deduplication using a processor device in a computing environment, an optimal calculation operation is applied in polynomial time to the matching segments for selecting a globally optimal subset of a set of matching segments according to overhead considerations for minimizing an overall size of a deduplicated file by determining a trade off between a time complexity and a space complexity.

Abstract: Methods, computer systems, and computer program products for processing data a computing environment are provided. The computer environment for data deduplication storage receives a plurality of write operations for deduplication storage of the data. The data is buffered in a plurality of buffers with overflow temporarily stored to a memory hierarchy when the data received for deduplication storage is sequential or non sequential. The data is accumulated and updated in the plurality of buffers per a data structure, the data structure serving as a fragment map between the plurality of buffers and a plurality of user file locations. The data is restructured in the plurality of buffers to form a complete sequence of a required sequence size. The data is provided as at least one stream to a stream-based deduplication algorithm for processing and storage.

Abstract: Systems and methods for managing resources for maintenance tasks in computing systems are provided. One system includes a controller and memory coupled to the controller, the memory configured to store a module. The controller, when executing the module, is configured to determine an amount of available resources for use by a plurality of maintenance tasks in a computing system and divide the available resources between the plurality of maintenance tasks based on a need for each maintenance task. One method includes determining, by a central controller, an amount of available resources for use by a plurality of maintenance tasks in a computing system and dividing the available resources between the plurality of maintenance tasks based on a need for each maintenance task. Computer storage mediums including a computer program product method for managing resources for maintenance tasks in computing systems are also provided.

Abstract: Various embodiments for replicating deduplicated data using a processor device are provided. A block of the deduplicated data, created in a source repository, is assigned a global block identification (id) unique in a grid set inclusive of the source repository. The global block id is generated using at least one unique identification value of the block, a containing grid of the grid set, and the source repository. The global block id is transmitted from the source repository to a target repository. If the target repository determines the global block id is associated with an existing block of the deduplicated data located within the target repository, the block is not received by the target repository during a subsequent replication process.

Abstract: Methods, computer systems, and computer program products for processing data a computing environment are provided. The computer environment for data deduplication storage receives a plurality of write operations for deduplication storage of the data. The data is buffered in a plurality of buffers with overflow temporarily stored to a memory hierarchy when the data received for deduplication storage is sequential or non sequential. The data is accumulated and updated in the plurality of buffers per a data structure, the data structure serving as a fragment map between the plurality of buffers and a plurality of user file locations. The data is restructured in the plurality of buffers to form a complete sequence of a required sequence size. The data is provided as at least one stream to a stream-based deduplication algorithm for processing and storage.

Abstract: Methods, computer systems, and computer program products for processing data a computing environment are provided. The computer environment for data deduplication storage receives a plurality of write operations for deduplication storage of the data. The data is buffered in a plurality of buffers with overflow temporarily stored to a memory hierarchy when the data received for deduplication storage is sequential or non sequential. The data is accumulated and updated in the plurality of buffers per a data structure, the data structure serving as a fragment map between the plurality of buffers and a plurality of user file locations. The data is restructured in the plurality of buffers to form a complete sequence of a required sequence size. The data is provided as at least one stream to a stream-based deduplication algorithm for processing and storage.

Abstract: Various embodiments for replicating deduplicated data using a processor device are provided. A block of the deduplicated data, created in a source repository, is assigned a global block identification (id) unique in a grid set inclusive of the source repository. The global block id is generated using at least one unique identification value of the block, a containing grid of the grid set, and the source repository. The global block id is transmitted from the source repository to a target repository. If the target repository determines the global block id is associated with an existing block of the deduplicated data located within the target repository, the block is not received by the target repository during a subsequent replication process.

Abstract: Method and apparatus are disclosed for a computer system including host computers and storage elements. The host computers may be configured into a host domain and the storage elements configured into a storage domain. The storage domain includes a plurality of primary storage devices and a secondary storage device. The secondary storage device may be coupled to a plurality of the primary storage devices through a network. In another embodiment, the secondary storage device may send and receive information from a heterogeneous set of host computers. In another embodiment, connections for transfer of data onto the secondary storage element are created automatically.