Abstract: A memory tier is established in a cluster system having a deduplicated file system. The memory tier includes memory pages configured as huge pages, where writes to the huge pages are exported in a device file that is outside of a user process namespace within which processes of the deduplicated file system run. At least a portion of metadata generated by the deduplicated file system is written to the memory tier. The portion of metadata includes an index of fingerprints corresponding to data segments stored by the deduplicated file system to a storage pool. A determination is made that an instance of the deduplicated file system has failed. A new instance of the deduplicated file system is started to recover file system services by loading the index of fingerprints from the device file.

Abstract: System identifies multiple data blocks in workload stored in slow access persistent storage, data blocks copied to fast access persistent storage, and, after speed of accessing workload satisfies threshold, copied data blocks that remained in fast access persistent storage. System annotates some remaining data blocks with cache label and derives features for some data blocks in workload, based on corresponding bits set and/or time stamp. System uses cache labels and features for some data blocks in workload to train machine-learning model to predict which data blocks in workload will remain in fast access persistent storage after workload access satisfies threshold. System derives features for data block requested from production workload.

Abstract: A memory tier is established in a cluster system having a deduplicated file system. The memory tier includes memory pages configured as huge pages, where writes to the huge pages are exported in a device file that is outside of a user process namespace within which processes of the deduplicated file system run. At least a portion of metadata generated by the deduplicated file system is written to the memory tier. The portion of metadata includes an index of fingerprints corresponding to data segments stored by the deduplicated file system to a storage pool. A determination is made that an instance of the deduplicated file system has failed. A new instance of the deduplicated file system is started to recover file system services by loading the index of fingerprints from the device file.

Abstract: A memory tier including persistent memory (PMEM) devices is established in nodes of a cluster system having a deduplicated file system. At least a portion of metadata generated by the deduplicated file system is persisted to the memory tier. The portion of metadata includes an index of fingerprints corresponding to data segments stored by the deduplicated file system to a storage pool. A determination is made that an instance of the deduplicated file system has failed. A new instance of the deduplicated file system is started to recover file system services by loading the index of fingerprints from the memory tier.

Abstract: Embodiments are described for prioritizing input/output (I/O) operations dispatched from a storage media to a host bus adapter in a network, by tagging, in an I/O tagging module, the I/O operations in a file system supporting applications generating the I/O operations, wherein all child I/O operations initiated from a parent I/O operation are tagged with a same unique tag ID; tracking a time of arrival of each I/O operation of the I/O operations; and dispatching, in a transactional I/O scheduler, all sibling I/O's of the parent I/O operation based on a unique tag ID for the sibling I/Os, a respective time of arrival of each of the sibling I/Os, and defined quality of service (QoS) requirements.

Abstract: One embodiment provides a computer implemented method of data compression including segmenting user data into data segments; deduplicating the data segments to form deduped data segments; compressing the deduped data segments into compression units using a hardware accelerator; packing the compression units into compression regions; and packing the compression regions into one or more containers.

Abstract: One embodiment provides a computer implemented method of balancing mixed workload performance including monitoring the compression and decompression workload at a hardware accelerator using the hardware accelerator quality of service (QoS) scheduler; monitoring the compression and decompression workload at a CPU using the CPU QoS scheduler; comparing the workload at the hardware accelerator and the workload at the CPU; and allocating tasks between the hardware accelerator and the CPU to obtain an optimal bandwidth at the hardware accelerator and the CPU.

Abstract: Embodiments are described for prioritizing input/output (I/O) operations dispatched to a solid-state device (SSD) cache in a network, by defining a maximum write I/O operation size for writing data to the SSD cache, splitting large write I/O operations into smaller write I/O operations, each with a size less than the maximum write I/O operation size, interleaving cache read I/O operations in between the smaller write I/O operations, and performing the cache read I/O operations and the smaller write I/O operations in an order created by the interleaving. The network may comprise a deduplication backup system storing data to storage media including the SSD cache.

Abstract: System identifies multiple data blocks in workload stored in slow access persistent storage, data blocks copied to fast access persistent storage, and, after speed of accessing workload satisfies threshold, copied data blocks that remained in fast access persistent storage. System annotates some remaining data blocks with cache label and derives features for some data blocks in workload, based on corresponding bits set and/or time stamp. System uses cache labels and features for some data blocks in workload to train machine-learning model to predict which data blocks in workload will remain in fast access persistent storage after workload access satisfies threshold. System derives features for data block requested from production workload.

Abstract: A method, apparatus, and system for decompressing data with a hardware compression/decompression accelerator is disclosed.

Abstract: Embodiments are described for prioritizing input/output (I/O) operations dispatched to a solid-state device (SSD) storage in a network, determining properties of the SSD storage based on SSD device type, where the SSD storage maintains a plurality of caches, receiving a plurality of SSD I/Os from an application I/O request comprising at least one of a cache read or a cache write operation, creating a respective queue for each SSD I/O of the SSD I/Os based on a type of SSD I/O, prioritizing the each SSD I/O based on the type of the SSD I/O and the properties of the SSD storage that is the target for a respective SSD I/O, and dispatching, in a transactional I/O SSD scheduler, all queued SSD I/Os to the SSD storage based on the prioritizing.

Abstract: A method, apparatus, and system for handling a failure of a hardware cryptography/compression accelerator is disclosed. The operations comprise: detecting that a hardware cryptography/compression accelerator at a first data storage system has failed; determining one or more failed cryptography and/or compression operation tasks that were submitted to the hardware cryptography/compression accelerator but were not completed due to the failure of the hardware cryptography/compression accelerator; and performing a remedial operation in response to the hardware cryptography/compression accelerator failure to prevent a systemic failure.

Abstract: One embodiment provides a computer implemented method of dynamically allocating compression jobs including monitoring compression performance at a plurality of compression levels within a hardware accelerator; comparing compression performance between the plurality of compression levels; and dynamically allocating compression jobs between the plurality of compression levels to achieve a guaranteed average performance.

Abstract: Embodiments are described for prioritizing input/output (I/O) operations dispatched to a solid-state device (SSD) cache in a network, by defining a maximum write I/O operation size for writing data to the SSD cache, splitting large write I/O operations into smaller write I/O operations, each with a size less than the maximum write I/O operation size, interleaving cache read I/O operations in between the smaller write I/O operations, and performing the cache read I/O operations and the smaller write I/O operations in an order created by the interleaving. The network may comprise a deduplication backup system storing data to storage media including the SSD cache.

Abstract: Embodiments are described for prioritizing input/output (I/O) operations dispatched to a solid-state device (SSD) storage in a network, determining properties of the SSD storage based on SSD device type, where the SSD storage maintains a plurality of caches, receiving a plurality of SSD I/Os from an application I/O request comprising at least one of a cache read or a cache write operation, creating a respective queue for each SSD I/O of the SSD I/Os based on a type of SSD I/O, prioritizing the each SSD I/O based on the type of the SSD I/O and the properties of the SSD storage that is the target for a respective SSD I/O, and dispatching, in a transactional I/O SSD scheduler, all queued SSD I/Os to the SSD storage based on the prioritizing.

Abstract: Embodiments are described for prioritizing input/output (I/O) operations dispatched from a storage media to a host bus adapter in a network, by tagging, in an I/O tagging module, the I/O operations in a file system supporting applications generating the I/O operations, wherein all child I/O operations initiated from a parent I/O operation are tagged with a same unique tag ID; tracking a time of arrival of each I/O operation of the I/O operations; and dispatching, in a transactional I/O scheduler, all sibling I/O's of the parent I/O operation based on a unique tag ID for the sibling I/Os, a respective time of arrival of each of the sibling I/Os, and defined quality of service (QoS) requirements.

Abstract: A method, apparatus, and system for decompressing data with a hardware compression/decompression accelerator is disclosed.

Abstract: One embodiment provides a computer implemented method of improving backup and restore performance including sending a compression job to a hardware accelerator using a compression thread; providing a callback pointer for the compression job; monitoring the hardware accelerator using a polling thread; calling the callback pointer to notify the compression thread when the hardware accelerator is available; and retrieving data from a destination buffer using the compression thread via a destination buffer pointer.

Abstract: A method, apparatus, and system for handling a failure of a hardware cryptography/compression accelerator is disclosed. The operations comprise: detecting that a hardware cryptography/compression accelerator at a first data storage system has failed; determining one or more failed cryptography and/or compression operation tasks that were submitted to the hardware cryptography/compression accelerator but were not completed due to the failure of the hardware cryptography/compression accelerator; and performing a remedial operation in response to the hardware cryptography/compression accelerator failure to prevent a systemic failure.

Abstract: One embodiment provides a computer implemented method of data compression including segmenting user data into data segments; deduplicating the data segments to form deduped data segments; compressing the deduped data segments into compression units using a hardware accelerator; packing the compression units into compression regions; and packing the compression regions into one or more containers.