Abstract: A computer-implemented method according to one embodiment includes identifying a data write to a specific position within a virtual address space, determining an entry within a metadata structure that corresponds to the specific position within the virtual address space, and adding state information associated with the data write to the entry within the metadata structure, the state information including a size of the data write within the virtual address space and an alignment of the data write within the virtual address space.

Abstract: A mechanism is provided in a data processing system comprising at least one processor and at least one memory. The at least one memory comprise instructions which are executed by the at least one processor and configure the processor to implement a read-ahead manager for adaptive read-ahead in log structured storage. The read-ahead manager determines a probability value P representing a probability to read into cache a temporal environment for a front-end read for a given segment in user space in a log structured storage. Responsive to performing a front-end read of a record of the given segment in the log structured storage, the read-ahead manager performs pre-fetch of the temporal environment for the record with probability P.

Abstract: Embodiments for efficient garbage collection in a data storage environment. In a storage system comprising multiple storage devices having respective sets of storage regions, at least one respective storage fragmentation threshold used to trigger a garbage collection operation is identified. The garbage collection operation is performed to reclaim data space in the storage system according to each of a block perspective and an area perspective. The block perspective performs the garbage collection operation on individual blocks of data and the area perspective performs the garbage collection operation on a plurality of the blocks in a respective storage region. The block perspective and the area perspective portions of the garbage collection operation are executed independently of one another.

Abstract: In one embodiment, a computer-implemented method for performing deduplication in conjunction with random read and write operations across a namespace that is divided into a plurality of disjoint regions is disclosed. The method includes: maintaining a metadata structure for each of the plurality of disjoint regions via the respective region manager, each metadata structure comprising metadata indicating a physical storage location of one or more data chunks associated with the respective region; and performing a deduplicated write operation. The deduplicated write operation includes: determining whether the disjoint regions comprise a second data chunk identical in content to a first data chunk; and in response to determining one or more of the plurality of disjoint regions other than the first region includes the second data chunk, establishing a reference from the first data chunk to the second data chunk. Corresponding systems and computer program products are also disclosed.

Abstract: An open address probing barrier is utilized in association with a memory container. A starting memory slot is calculated for an item to be found in the memory container. Responsive to the item failing to occupy the starting memory slot, a first predetermined number of memory slots immediately following the starting memory slot are probed. Responsive to the item occupying one of those memory slots, the item is returned to the application. Responsive to the item failing to occupy one of those memory slots, a determination is made as to whether a barrier bit has been set in association with the last of the first predetermined number of memory slots. If so, at least a portion of the memory container is probed for the item and, if found, the item is returned to the application. If the barrier bit is not set, a notification is returned to the application.

Abstract: A mechanism is provided in a data processing system comprising at least one processor and at least one memory. The at least one memory comprise instructions which are executed by the at least one processor and configure the processor to implement a read-ahead manager for adaptive read-ahead in log structured storage. The read-ahead manager determines a probability value P representing a probability to read into cache a temporal environment for a front-end read for a given segment in user space in a log structured storage. Responsive to performing a front-end read of a record of the given segment in the log structured storage, the read-ahead manager performs pre-fetch of the temporal environment for the record with probability P.

Abstract: A mechanism is provided for dispersed location-based data storage. A request is received to write a data file to a referrer memory region in a set of memory regions. For each data chunk of the data file, responsive to a comparison of a hash value for the data chunk to other hash values for other stored data chunks referenced in the referrer memory region indicating that the data chunk fails to exist in the referrer memory region, responsive to the data chunk existing in another memory region in the set of memory regions, responsive to the memory region failing to be one of the predetermined number N of owner memory regions associated with the referrer memory region, and responsive to the predetermined number N of owner memory regions failing to have been met, a reference to the data chunk is stored in the referrer memory region.

Abstract: A mechanism is provided in a data processing system comprising at least one processor and at least one memory. The at least one memory comprise instructions which are executed by the at least one processor and configure the processor to implement a read-ahead manager for adaptive read-ahead in log structured storage. The read-ahead manager determines a probability value P representing a probability to read into cache a temporal environment for a front-end read for a given segment in user space in a log structured storage. Responsive to performing a front-end read of a record of the given segment in the log structured storage, the read-ahead manager performs pre-fetch of the temporal environment for the record with probability P.

Abstract: An open address probing barrier is utilized in association with a memory container. A starting memory slot is calculated for an item to be found in the memory container. Responsive to the item failing to occupy the starting memory slot, a first predetermined number of memory slots immediately following the starting memory slot are probed. Responsive to the item occupying one of those memory slots, the item is returned to the application. Responsive to the item failing to occupy one of those memory slots, a determination is made as to whether a barrier bit has been set in association with the last of the first predetermined number of memory slots. If so, at least a portion of the memory container is probed for the item and, if found, the item is returned to the application. If the barrier bit is not set, a notification is returned to the application.

Abstract: A computer program product, according to one embodiment, includes a computer readable storage medium having program instructions embodied therewith. The computer readable storage medium is not a transitory signal per se. The program instructions are readable and/or executable by a processor to cause the processor to perform a method which includes: receiving a request to delete a volume stored in one or more regions in physical space of a storage system; determining whether at least one of the regions having at least a portion of the volume includes reclaimable space; deleting the portion of the volume from the at least one region having the reclaimable space in response to determining that at least one of the regions having at least a portion of the volume includes reclaimable space; and failing the received request to delete the volume in response to determining that none of the regions include reclaimable space.

Abstract: Links between data regions are intelligently formed, based on up-to-date popularity statistics, including a number of times a particular one of the data regions was a target for a potential link with another one of the data regions. An owner manages a popularity index incorporating the popularity statistics, where over time, the popularity index is subjected to an aging mechanism pursuant to a decaying algorithm. An owner one of the data regions decides whether to accept the one of the intelligently formed links based on at least one of a plurality of predetermined factors. If the one of the intelligently formed links is accepted by the owner one of the data regions, a potential referrer creates one of the data regions using one of the intelligently formed links. If the one of the intelligently formed links is rejected by the owner one of the data regions, data is written.

Abstract: In one embodiment, a computer-implemented method for performing deduplication in conjunction with random read and write operations across a namespace that is divided into a plurality of disjoint regions is disclosed. The method includes: maintaining a metadata structure for each of the plurality of disjoint regions via the respective region manager, each metadata structure comprising metadata indicating a physical storage location of one or more data chunks associated with the respective region; and performing a deduplicated write operation. The deduplicated write operation includes: determining whether the disjoint regions comprise a second data chunk identical in content to a first data chunk; and in response to determining one or more of the plurality of disjoint regions other than the first region includes the second data chunk, establishing a reference from the first data chunk to the second data chunk. Corresponding systems and computer program products are also disclosed.

Abstract: Embodiments for optimizing dual-layered data compression in a storage environment. In a data storage system having a primary compressor and a secondary compressor, the primary compressor is selectively used to perform a first one of a plurality of actions on Input/Output (I/O) data while a second one of the plurality of actions is performed on the I/O data by the secondary compressor, thereby reducing latency and improving an overall compression performance while processing the I/O data.

Abstract: In one embodiment, a computer program product is configured for performing deduplication in conjunction with random read and write operations across a namespace divided into a plurality of disjoint regions. The computer program product includes a computer readable storage medium having program instructions embodied therewith, where the computer readable storage medium is not a transitory signal per se. The program instructions are executable by a computer to cause the computer to perform a method including: maintaining a metadata structure for each of the plurality of disjoint regions via the respective region manager(s) of the plurality of disjoint regions, each metadata structure comprising metadata indicating a physical storage location of one or more data chunks associated with the respective region; and performing, by the computer, a deduplicated write operation of a first data chunk in a first region of the plurality of disjoint regions.

Abstract: Various embodiments for optimizing deduplication in a computing storage environment by a processor. Links between data regions are intelligently formed, based on up-to-date popularity statistics, including a number of times a particular one of the data regions was a target for a potential link with another one of the data regions.

Abstract: Various embodiments for identifying data in a data deduplication system, by a processor device, are provided. In one embodiment, a method comprises efficiently identifying duplicate data in the data deduplication system by identifying fingerprint matches using a direct inter-region fingerprint lookup to search for the fingerprint matches in at least one of a plurality of metadata regions, the direct inter-region fingerprint lookup supplementing a central fingerprint index.

Abstract: In one embodiment, a computer program product is configured for performing deduplication in conjunction with random read and write operations across a namespace divided into a plurality of disjoint regions. The computer program product includes a computer readable storage medium having program instructions embodied therewith, where the computer readable storage medium is not a transitory signal per se. The program instructions are executable by a computer to cause the computer to perform a method including: maintaining a metadata structure for each of the plurality of disjoint regions via the respective region manager(s) of the plurality of disjoint regions, each metadata structure comprising metadata indicating a physical storage location of one or more data chunks associated with the respective region; and performing, by the computer, a deduplicated write operation of a first data chunk in a first region of the plurality of disjoint regions.

Abstract: Calculating fingerprints for each one of a multiplicity of alignment combinations of fixed-size deduplication data blocks and comparing each of the fingerprints to stored deduplicated data fingerprints in a lookup database for determining a preferred deduplication data block alignment. A deduplication data block comprises each of the fixed-size deduplication data blocks.

Abstract: Methods, computing systems and computer program products implement embodiments of the present invention that include receiving a request to create a storage entity on a storage system, the storage entity including data and metadata, the metadata used to manage the storage entity. Upon receiving the request, multiple metadata attributes are identified for the metadata, and for each given identified metadata attribute, a respective metadata region is created on the storage system, and a subset of the metadata having the given metadata attribute is stored to the respective metadata region. Finally, a data region is created on the storage system, and the data is stored to the data region.

Abstract: Various embodiments for identifying data in a data deduplication system, by a processor device, are provided. In one embodiment, a method comprises efficiently identifying duplicate data in the data deduplication system by identifying fingerprint matches using a direct inter-region fingerprint lookup to search for the fingerprint matches in at least one of a plurality of metadata regions, the direct inter-region fingerprint lookup supplementing a central fingerprint index.