Abstract: Methods that can provide distributed data deduplication reference counting are disclosed herein. One method includes generating, by a processor, a secondary reference in response to a determination that a data chunk is a copy of the primary reference in which the primary reference includes a pointer to an existing data chunk stored on a storage device. The method further includes incrementing a secondary reference counter on the primary reference in response to generating the secondary reference. Apparatus and computer program products that can include, perform, and/or implement the methods are also disclosed herein.

Abstract: A method and system may reconstruct data in a smart storage array where upon detection of data inconsistency, an application of the system is notified and affected data strips are rebuilt. When an initiator detects stripe corruption, the initiator may report the strip corruption to storage. The storage may lock the strip for I/O operations. Initiators may determine recovery scenarios for rebuilding the data strips and send the scenarios to storage. At storage, initiator replies may be collected, and a rebuild option with the highest votes may be automatically determined for a reconstruction operation. Once the reconstruction operation is completed, the rebuilt stripe may be unlocked and data operations may recommence.

Abstract: A mechanism is provided for dispersed location-based data storage. A request is received to write a data file to a referrer memory region in a set of memory regions. For each data chunk of the data file, responsive to a comparison of a hash value for the data chunk to other hash values for other stored data chunks referenced in the referrer memory region indicating that the data chunk fails to exist in the referrer memory region, responsive to the data chunk existing in another memory region in the set of memory regions, responsive to the memory region failing to be one of the predetermined number N of owner memory regions associated with the referrer memory region, and responsive to the predetermined number N of owner memory regions failing to have been met, a reference to the data chunk is stored in the referrer memory region.

Abstract: A mechanism is provided for dispersed location-based data storage. A request is received to write a data file to a referrer memory region in a set of memory regions. For each data chunk of the data file, responsive to a comparison of a hash value for the data chunk to other hash values for other stored data chunks referenced in the referrer memory region indicating that the data chunk fails to exist in the referrer memory region, responsive to the data chunk existing in another memory region in the set of memory regions, responsive to the memory region failing to be one of the predetermined number N of owner memory regions associated with the referrer memory region, and responsive to the predetermined number N of owner memory regions failing to have been met, a reference to the data chunk is stored in the referrer memory region.

Abstract: A method and system may reconstruct data in a smart storage array where upon detection of data inconsistency, an application of the system is notified and affected data strips are rebuilt. When an initiator detects stripe corruption, the initiator may report the strip corruption to storage. The storage may lock the strip for I/O operations. Initiators may determine recovery scenarios for rebuilding the data strips and send the scenarios to storage. At storage, initiator replies may be collected, and a rebuild option with the highest votes may be automatically determined for a reconstruction operation. Once the reconstruction operation is completed, the rebuilt stripe may be unlocked and data operations may recommence.

Abstract: Embodiments for reducing rebuild time in a computing storage environment in by a processor. One or more disk drive failures in a Redundant Array of Independent Disks (RAID)-6 may be rebuilt by holding at least three parity strips per stripe while using one or more of the at least three parity strips according the one or more disk drive failures.

Abstract: A computer-implemented method according to one embodiment includes identifying a data write to a specific position within a virtual address space, determining an entry within a metadata structure that corresponds to the specific position within the virtual address space, and adding state information associated with the data write to the entry within the metadata structure, the state information including a size of the data write within the virtual address space and an alignment of the data write within the virtual address space.

Abstract: A computer-implemented method according to one embodiment includes receiving data, storing chunks of the data in fixed size blocks in physical storage, and deduplicating a copy of the data. The storing includes splitting some of the data chunks into portions for filling the fixed size blocks. Metadata is stored with the data chunks for chaining the associated portions of the split data chunks together. The deduplicating including creating a pointer to a first of the data chunks. A computer program product for utilizing free space in physical storage, according to another embodiment includes a computer readable storage medium having program instructions embodied therewith. The computer readable storage medium is not a transitory signal per se. The program instructions are readable and/or executable by a computer to cause the computer to perform the foregoing method.

Abstract: A mechanism is provided in a data processing system comprising at least one processor and at least one memory. The at least one memory comprise instructions which are executed by the at least one processor and configure the processor to implement a read-ahead manager for adaptive read-ahead in log structured storage. The read-ahead manager determines a probability value P representing a probability to read into cache a temporal environment for a front-end read for a given segment in user space in a log structured storage. Responsive to performing a front-end read of a record of the given segment in the log structured storage, the read-ahead manager performs pre-fetch of the temporal environment for the record with probability P.

Abstract: A computer-implemented method according to one embodiment includes determining resource usage of at least a first module in a grid storage system having multiple modules and approximately equal resource usage across the multiple modules of the grid storage system. The computer-implemented method further includes determining a garbage collection cost in the grid storage system by stopping garbage collection in a second of the modules of the grid storage system, determining a resource usage in the second module upon stopping the garbage collection, and comparing the resource usage in the second module to the resource usage of the at least the first module. The method further includes adjusting an amount of garbage collection based on both the garbage collection cost and the resource usage.

Abstract: A computer-implemented method, according to one embodiment, includes: determining which of a plurality of fingerprints correspond to a copy of user data stored in one or more logical volumes at a storage location, and which of the plurality of fingerprints correspond to a pointer that points to a copy of user data stored in the one or more logical volumes at the storage location. A number of unique fingerprints is determined, and a number of the copies of user data stored in the one or more logical volumes is determined. The number of the copies of user data and the number of unique fingerprints are used to calculate an actual deduplication efficiency value. The number of the copies of user data and the number of unique fingerprints are used to calculate a potential deduplication efficiency value. The actual deduplication efficiency value is compared to the potential deduplication efficiency value.

Abstract: Embodiments for reducing rebuild time in a computing storage environment in by a processor. One or more disk drive failures in a Redundant Array of Independent Disks (RAID)-6 may be rebuilt by holding at least three parity strips per stripe while using one or more of the at least three parity strips according the one or more disk drive failures.

Abstract: A computer program product, according to one embodiment, includes a computer readable storage medium having program instructions embodied therewith. The computer readable storage medium is not a transitory signal per se. The program instructions are readable and/or executable by a processor to cause the processor to perform a method which includes: receiving a request to delete a volume stored in one or more regions in physical space of a storage system; determining whether at least one of the regions having at least a portion of the volume includes reclaimable space; deleting the portion of the volume from the at least one region having the reclaimable space in response to determining that at least one of the regions having at least a portion of the volume includes reclaimable space; and failing the received request to delete the volume in response to determining that none of the regions include reclaimable space.

Abstract: A computer-implemented method according to one embodiment includes identifying a data write to a specific position within a virtual address space, determining an entry within a metadata structure that corresponds to the specific position within the virtual address space, and adding state information associated with the data write to the entry within the metadata structure, the state information including a size of the data write within the virtual address space and an alignment of the data write within the virtual address space.

Abstract: A computer-implemented method according to one embodiment includes determining resource usage of at least a first module in a grid storage system having multiple modules and approximately equal resource usage across the multiple modules of the grid storage system. The computer-implemented method further includes determining a garbage collection cost in the grid storage system by stopping garbage collection in a second of the modules of the grid storage system, determining a resource usage in the second module upon stopping the garbage collection, and comparing the resource usage in the second module to the resource usage of the at least the first module. The method further includes adjusting an amount of garbage collection based on both the garbage collection cost and the resource usage.

Abstract: Methods that can provide distributed data deduplication reference counting are disclosed herein. One method includes generating, by a processor, a secondary reference in response to a determination that a data chunk is a copy of the primary reference in which the primary reference includes a pointer to an existing data chunk stored on a storage device. The method further includes incrementing a secondary reference counter on the primary reference in response to generating the secondary reference. Apparatus and computer program products that can include, perform, and/or implement the methods are also disclosed herein.

Abstract: Embodiments for efficient garbage collection in a data storage environment. In a storage system comprising multiple storage devices having respective sets of storage regions, at least one respective storage fragmentation threshold used to trigger a garbage collection operation is identified. The garbage collection operation is performed to reclaim data space in the storage system according to each of a block perspective and an area perspective. The block perspective performs the garbage collection operation on individual blocks of data and the area perspective performs the garbage collection operation on a plurality of the blocks in a respective storage region. The block perspective and the area perspective portions of the garbage collection operation are executed independently of one another.

Abstract: A computer-implemented method according to one embodiment includes identifying a data write to a specific position within a virtual address space, determining an entry within a metadata structure that corresponds to the specific position within the virtual address space, and adding state information associated with the data write to the entry within the metadata structure, the state information including a size of the data write within the virtual address space and an alignment of the data write within the virtual address space.

Abstract: A computer-implemented method, according to one embodiment, includes: determining which of a plurality of fingerprints correspond to a copy of user data stored in one or more logical volumes at a storage location, and which of the plurality of fingerprints correspond to a pointer that points to a copy of user data stored in the one or more logical volumes at the storage location. A number of unique fingerprints is determined, and a number of the copies of user data stored in the one or more logical volumes is determined. The number of the copies of user data and the number of unique fingerprints are used to calculate an actual deduplication efficiency value. The number of the copies of user data and the number of unique fingerprints are used to calculate a potential deduplication efficiency value. The actual deduplication efficiency value is compared to the potential deduplication efficiency value.

Abstract: A computer-implemented method according to one embodiment includes identifying a data write to a specific position within a virtual address space, determining an entry within a metadata structure that corresponds to the specific position within the virtual address space, and adding state information associated with the data write to the entry within the metadata structure, the state information including a size of the data write within the virtual address space and an alignment of the data write within the virtual address space.