Abstract: Various embodiments for repository management in a data deduplication system, by a processor device, are provided. Metadata of an inode structure of an entire pre-allocated file system is captured, exported, and compressed from an existing deduplication appliance, the pre-allocated file system comprising a fully padded file system. The exported and compressed metadata of the pre-allocated file system is decompressed and imported into a data deduplication repository of a new deduplication appliance having an identical file system size as within the existing deduplication appliance, to initially configure or subsequently scale the inode structure of a file system of the data deduplication repository of the new deduplication appliance efficiently.

Abstract: Various embodiments for managing data in a data storage having data deduplication. A back reference data structure is configured for user data segments as a mechanism to identify an affected storage block to which information in the back reference data structure refers. The back reference data structure is initialized such that a resolution of the back reference data structure diminishes as a number of the user data segments referencing the affected storage block increases.

Abstract: Various embodiments for repository management in a data deduplication system, by a processor device, are provided. Metadata of an inode structure of an entire pre-allocated file system is captured, exported, and compressed from an existing deduplication appliance, the pre-allocated file system comprising a fully padded file system. The exported and compressed metadata of the pre-allocated file system is decompressed and imported into a data deduplication repository of a new deduplication appliance having an identical file system size as within the existing deduplication appliance, to initially configure or subsequently scale the inode structure of a file system of the data deduplication repository of the new deduplication appliance efficiently.

Abstract: Various embodiments for repository management in a data deduplication system, by a processor device, are provided. Metadata of a pre-allocated file system is captured and exported. The exported metadata is then imported into a data deduplication repository for configuring the data deduplication repository with minimum overhead.

Abstract: Various embodiments for repository management in a data deduplication system, by a processor device, are provided. Metadata of a pre-allocated file system is captured and exported. The exported metadata is then imported into a data deduplication repository for configuring the data deduplication repository with minimum overhead.

Abstract: For efficient calculation of both similarity search values and boundaries of digest blocks in data deduplication, input data is partitioned into chunks, and for each chunk a set of rolling hash values is calculated. A single linear scan of the rolling hash values is used to produce both similarity search values and boundaries of the digest blocks of the chunk. The rolling hash values are used to contribute to the calculation of the similarity search values and to the calculation of the boundaries of the digest blocks.

Abstract: For digest retrieval based on similarity search in deduplication processing in a data deduplication system using a processor device in a computing environment, input data is partitioned into fixed sized data chunks. Similarity elements and digest block boundaries and digest values are calculated for each of the fixed sized data chunks. Matching similarity elements are searched for in a search structure containing the similarity elements for each of the fixed sized data chunks in a repository of data. Positions of similar data are located in the repository. The positions of the similar data are used to locate and load into the memory stored digest values and corresponding stored digest block boundaries of the similar data in the repository. The digest values and the corresponding digest block boundaries of the input data are matched with the stored digest values and the corresponding stored digest block boundaries to find data matches.

Abstract: Various embodiments for managing data in a data storage having data deduplication. A back reference data structure is configured for user data segments as a mechanism to identify an affected storage block to which information in the back reference data structure refers. The back reference data structure is initialized such that a resolution of the back reference data structure diminishes as a number of the user data segments referencing the affected storage block increases.

Abstract: For efficient calculation of both similarity search values and boundaries of digest blocks in data deduplication, input data is partitioned into chunks, and for each chunk a set of rolling hash values is calculated. A single linear scan of the rolling hash values is used to produce both similarity search values and boundaries of the digest blocks of the chunk. The rolling hash values are used to contribute to the calculation of the similarity search values and to the calculation of the boundaries of the digest blocks.

Abstract: For efficient calculation of both similarity search values and boundaries of digest blocks in data deduplication, input data is partitioned into chunks, and for each chunk a set of rolling hash values is calculated. A single linear scan of the rolling hash values is used to produce both similarity search values and boundaries of the digest blocks of the chunk.

Abstract: A deduplication storage capacity is estimated as a function of an expected deduplication ratio, the expected deduplication ratio being a combined average of a current deduplication ratio and a configured deduplication ratio, the current deduplication ratio depending on the data currently stored in the deduplication storage, and the configured deduplication ratio being an estimate made at a configuration stage of the deduplication computing storage environment.

Abstract: For digest retrieval based on similarity search in deduplication processing in a data deduplication system using a processor device in a computing environment, input data is partitioned into fixed sized data chunks. Similarity elements and digest block boundaries and digest values are calculated for each of the fixed sized data chunks. Matching similarity elements are searched for in a search structure containing the similarity elements for each of the fixed sized data chunks in a repository of data. Positions of similar data are located in the repository. The positions of the similar data are used to locate and load into the memory stored digest values and corresponding stored digest block boundaries of the similar data in the repository. The digest values and the corresponding digest block boundaries of the input data are matched with the stored digest values and the corresponding stored digest block boundaries to find data matches.

Abstract: For efficient calculation of both similarity search values and boundaries of digest blocks in data deduplication, input data is partitioned into chunks, and for each chunk a set of rolling hash values is calculated. A single linear scan of the rolling hash values is used to produce both similarity search values and boundaries of the digest blocks of the chunk.

Abstract: A deduplication storage capacity is estimated as a function of an expected deduplication ratio, the expected deduplication ratio being a combined average of a current deduplication ratio and a configured deduplication ratio, the current deduplication ratio depending on the data currently stored in the deduplication storage, and the configured deduplication ratio being an estimate made at a configuration stage of the deduplication computing storage environment.

Abstract: A data storage system including at least one storage controller having a first color policy and operative to store data onto a first data storage unit at a primary site as part of a current color of the primary site, at least one storage controller having a second color policy and operative to store data onto a second data storage unit at the primary site as part of the current color, and a color control node operative to provide each of the controllers with new color information while maintaining the integrity of dependent writes across color boundaries.

Abstract: A deduplication storage capacity is estimated as a function of an expected deduplication ratio, the expected deduplication ratio being a combined average of a current deduplication ratio and a configured deduplication ratio, the current deduplication ratio depending on the data currently stored in the deduplication storage, and the configured deduplication ratio being an estimate made at a configuration stage of the deduplication computing storage environment.

Abstract: A color control node includes an interface for communicating with multiple storage controllers, wherein the storage controllers maintain a primary storage system at a primary site and a secondary storage system at a secondary site; and wherein the storage controllers maintain a current color and associate all writes with the current color without polling the color control node. The color control node also includes operational capability for issuing a polling command to instruct the storage controllers to poll the color control node for the current color prior to associating each write with a new color; receiving an acknowledgment of receipt of the polling command; changing the current color to a new color responsive to receiving the acknowledgment; issuing a storage command to the storage controllers indicating the new color; and instructing each storage controller to cease polling the color control node for the current color.

Abstract: Various embodiments for capacity management in a deduplication computing storage environment by a processor device are provided. A deduplication storage capacity is estimated as a function of an expected deduplication ratio, the expected deduplication ratio being a combined average of a current deduplication ratio and a configured deduplication ratio, the current deduplication ratio depending on the data currently stored in the deduplication storage, and the configured deduplication ratio being an estimate made at a configuration stage of the deduplication computing storage environment.

Abstract: A data storage system including at least one storage controller having a first color policy and operative to store data onto a first data storage unit at a primary site as part of a current color of the primary site, at least one storage controller having a second color policy and operative to store data onto a second data storage unit at the primary site as part of the current color, and a color control node operative to provide each of the controllers with new color information while maintaining the integrity of dependent writes across color boundaries.

Abstract: A color control node includes an interface for communicating with multiple storage controllers, wherein the storage controllers maintain a primary storage system at a primary site and a secondary storage system at a secondary site; and wherein the storage controllers maintain a current color and associate all writes with the current color without polling the color control node. The color control node also includes operational capability for issuing a polling command to instruct the storage controllers to poll the color control node for the current color prior to associating each write with a new color; receiving an acknowledgment of receipt of the polling command; changing the current color to a new color responsive to receiving the acknowledgment; issuing a storage command to the storage controllers indicating the new color; and instructing each storage controller to cease polling the color control node for the current color.