Abstract: Methods, computing systems and computer program products implement embodiments of the present invention that include configuring multiple servers coupled to a network as a software defined storage (SDS) grid. A first given server receives, via the network, an input/output (I/O) request from a host computer, and determines a location of data associated with the I/O request. In some embodiments, each of the servers maintains a local grid data map that store locations for all data managed by the SDS grid. Upon identifying, in its respective local grid data map, that a second given server is configured to process the I/O request, the first given server forwards the I/O request to the second given server for processing, and upon receiving a result of the I/O request from the second given server, the first given server conveys the result of the I/O request to the host computer.

Abstract: Various embodiments for efficient data transfer in a Remote Direct Memory Access (RDMA) operation by a memory device. A file protocol header of the data is replaced with a block protocol header. The block protocol header enables transfer over a direct data path through a transport layer from a source to a target without outside manipulation by a central processing unit (CPU).

Abstract: For efficient serialization of journal data using a processor device, serialization is combined with journaling of data into one data storage system on a write operation with subsequent consolidation for serialization of the journal data.

Abstract: Methods, computing systems and computer program products implement embodiments of the present invention that include configuring a storage system to store multiple storage entities for access by one or more host computers in communication with the storage system, and specifying a compression condition including a minimum compression ratio. The storage system can then estimate an expected compression ratio for a given storage entity, compress the given storage entity upon the expected compression ratio meeting the compression condition, and provide, to a given host computer, access to the compressed given storage entity.

Abstract: A Huffman cache is used to hold Huffman dictionaries that are changeable for dynamically selecting literal frequencies that are similar, wherein the Huffman cache is a data storage cache.

Abstract: A computer-implemented method includes receiving a set of representative machine image regions for a computing environment wherein the set of representative machine image regions collectively comprise a set of representative image chunks. The method also includes generating a fingerprint for each representative image chunk within the set of representative image chunks to produce a set of representative fingerprints, generating a fingerprint for selected image chunks within a measured machine image region to produce a set of sampled fingerprints, and determining a deduplication metric for the measured machine image region based on the representative fingerprints and the sampled fingerprints. A corresponding computer program product and computer system are also disclosed herein.

Abstract: Identification of data candidates for data processing is performed in real time by a processor device in a computing environment. Data candidates are sampled for performing a classification-based compression upon the data candidates. A heuristic is computed on a randomly selected data sample from the data candidate, the heuristic computed by, for each one of the data classes, calculating an expected number of characters to be in a data class, calculating an expected number of characters that will not belong to a predefined set of the data classes, and calculating an actual number of the characters for each of the data classes and the non-classifiable data.

Abstract: Systems and methods for compressing a raw logical data object (201) for storage in a storage device operable with at least one storage protocol, creating, reading, writing, optimizatic in and restoring thereof. Compressing the raw logical data object (201) comprises creating in the storage device a compressed logical data object (203) comprising a header (204) and one or more allocated compressed sections with predefined size (205-1-205-2); compressing one or more sequentially obtained chunks of raw data (202-1-202-6) corresponding to the raw logical data object (201) thus giving rise to the compressed data chunks (207-1-207-6); and sequentially accommodating the processed data chunks into: said compressed sections (205-1-205-2) in accordance with an order said chunks received, wherein said compressed sections serve as atomic elements of compression/decompression operations during input/output transactions on the logical data object.

Abstract: In one embodiment, a computer-implemented method for performing deduplication in conjunction with random read and write operations across a namespace that is divided into a plurality of disjoint regions is disclosed. The method includes: maintaining a metadata structure for each of the plurality of disjoint regions via the respective region manager, each metadata structure comprising metadata indicating a physical storage location of one or more data chunks associated with the respective region; and performing a deduplicated write operation. The deduplicated write operation includes: determining whether the disjoint regions comprise a second data chunk identical in content to a first data chunk; and in response to determining one or more of the plurality of disjoint regions other than the first region includes the second data chunk, establishing a reference from the first data chunk to the second data chunk. Corresponding systems and computer program products are also disclosed.

Abstract: Systems and methods for compressing a raw logical data object (201) for storage in a storage device operable with at least one storage protocol, creating, reading, writing, optimizatic in and restoring thereof. Compressing the raw logical data object (201) comprises creating in the storage device a compressed logical data object (203) comprising a header (204) and one or more allocated compressed sections with predefined size (205-1-205-2); compressing one or more sequentially obtained chunks of raw data (202-1-202-6) corresponding to the raw logical data object (201) thus giving rise to the compressed data chunks (207-1-207-6); and sequentially accommodating the processed data chunks into: said compressed sections (205-1-205-2) in accordance with an order said chunks received, wherein said compressed sections serve as atomic elements of compression/decompression operations during input/output transactions on the logical data object.

Abstract: Methods, systems, and computer storage mediums including a computer program product for managing data in a computing network are provided. One method includes registering a plurality of buffers with a work queue in a server and assigning ownership to each of the plurality of buffers to a different working thread in a plurality of working threads. The method further includes continuously polling, by a polling thread, the work queue to determine when work requests are received by the work queue, upon receipt of each work request, and performing each work request on each respective associated buffer by a working thread among the plurality of working threads that owns each respective associated buffer. One system includes a processor for performing the above method and one computer storage medium includes computer code for performing the above method.

Abstract: Methods, systems, and computer program product embodiments for controlling congestion in a storage grid, by a processor device, are provided. In a storage grid, a storage request transmit queue length of storage requests submitted by a plurality of nodes is monitored, the storage request transmit queue comprising a queue holding the storage requests unable to be transmitted through a current transmission control protocol/internet protocol (TCP/IP) socket buffer of the storage grid. Upon reaching at least one of a certain threshold, a watermarked message indicating a specific one of the plurality of nodes is congested is transmitted to upper layer grid nodes in a hierarchy of the plurality of nodes transmitting the storage requests, and the upper layer grid nodes each alter the storage requests based upon the watermarked message.

Abstract: In a compression processing storage system, using a pool of encryption processing cores, the encryption processing cores are assigned to process either encryption operations, decryption operations, and decryption and encryption operations, that are scheduled for processing. A maximum number of the encryption processing cores are set for processing only the decryption operations, thereby lowering a decryption latency. A minimal number of the encryption processing cores are allocated for processing the encryption operations, thereby increasing encryption latency. The encryption operations, the decryption operations, and the decryption and encryption operations are scheduled between the pool of the plurality of processing cores according to a thread weight value (TWV) that is assigned to each one of the plurality of processing cores having a difference in processing power.

Abstract: In one embodiment, a computer-implemented method for performing deduplication in conjunction with random read and write operations across a namespace that is divided into a plurality of disjoint regions is disclosed. The method includes: maintaining a metadata structure for each of the plurality of disjoint regions via the respective region manager, each metadata structure comprising metadata indicating a physical storage location of one or more data chunks associated with the respective region; and performing a deduplicated write operation. The deduplicated write operation includes: determining whether the disjoint regions comprise a second data chunk identical in content to a first data chunk; and in response to determining one or more of the plurality of disjoint regions other than the first region includes the second data chunk, establishing a reference from the first data chunk to the second data chunk. Corresponding systems and computer program products are also disclosed.

Abstract: Methods, systems, and computer storage mediums including a computer program product for managing data in a computing network are provided. One method includes registering a plurality of buffers with a work queue in a server and assigning ownership to each of the plurality of buffers to a different working thread in a plurality of working threads. The method further includes continuously polling, by a polling thread, the work queue to determine when work requests are received by the work queue, upon receipt of each work request, and performing each work request on each respective associated buffer by a working thread among the plurality of working threads that owns each respective associated buffer. One system includes a processor for performing the above method and one computer storage medium includes computer code for performing the above method.

Abstract: In one embodiment, a computer program product is configured for performing deduplication in conjunction with random read and write operations across a namespace divided into a plurality of disjoint regions. The computer program product includes a computer readable storage medium having program instructions embodied therewith, where the computer readable storage medium is not a transitory signal per se. The program instructions are executable by a computer to cause the computer to perform a method including: maintaining a metadata structure for each of the plurality of disjoint regions via the respective region manager(s) of the plurality of disjoint regions, each metadata structure comprising metadata indicating a physical storage location of one or more data chunks associated with the respective region; and performing, by the computer, a deduplicated write operation of a first data chunk in a first region of the plurality of disjoint regions.

Abstract: In a compression processing storage system, using a pool of compression cores, the compression cores are assigned to process either compression operations, decompression operations, or decompression and compression operations, which are scheduled for processing. Only decompression operations are assigned to the plurality of compression cores having an idle status, where the idle status represents those of the plurality of compression cores that have yet to process any of the compression operations, decompression operations, and decompression and compression operations during a predetermined threshold period.

Abstract: Methods, systems, and computer storage mediums including a computer program product for managing data in a computing network are provided. One method includes registering a plurality of buffers with a work queue in a server and assigning ownership to each of the plurality of buffers to a different working thread in a plurality of working threads. The method further includes continuously polling, by a polling thread, the work queue to determine when work requests are received by the work queue, upon receipt of each work request, determining which buffer among the plurality of buffers each work request is associated, and performing each work request on each respective associated buffer by a working thread among the plurality of working threads that owns each respective associated buffer. One system includes a processor for performing the above method and one computer storage medium includes computer code for performing the above method.

Abstract: A detection learning module is used for enabling and/or disabling real-time compression detection by maintaining a history of real-time compression detection success for sampled data. The enabling or disabling of the real-time compression detection is based on a detection benefit function derived from a set of calculated heuristics indicating the real-time compression detection success on input streams. The detection benefit function is calculated based on at least one heuristic score.

Abstract: In a compression processing storage system, using a pool of encryption processing cores, the encryption processing cores are assigned to process either encryption operations, decryption operations, and decryption and encryption operations, that are scheduled for processing. A maximum number of the encryption processing cores are set for processing only the decryption operations, thereby lowering a decryption latency. A minimal number of the encryption processing cores are allocated for processing the encryption operations, thereby increasing encryption latency. The encryption operations, the decryption operations, and the decryption and encryption operations are scheduled between the pool of the plurality of processing cores according to a thread weight value (TWV) that is assigned to each one of the plurality of processing cores having a difference in processing power.