Abstract: A method for managing tasks in a storage system, the method may include: (a) obtaining, by a scheduler, a shared budget for background storage tasks and foreground storage tasks; (b) obtaining, by the scheduler, a background budget for background storage tasks; wherein the background budget is a fraction of the shared budget; (c) allocating, by the scheduler, resources to pending storage tasks according to the shared budget and the background budget; wherein the allocating comprises (i) allocating the shared budget while prioritizing foreground storage tasks over background storage tasks; and (ii) allocating the background budget to background storage tasks; and (d) participating, by the scheduler, in executing of storage tasks according to the allocation.

Abstract: A method for low-latency data mirroring in a storage system, the method may include receiving, by a compute node of the storage system, a read request for reading a data unit stored at a primary storage block of the storage system; determining, by the compute node and based on primary storage block metadata, whether a content of the primary storage block is consistent; and responding to the determining of whether the content of the primary storage block is consistent. The responding may include (a) when determining that the primary storage block is inconsistent—copying the data unit from the primary storage block to a secondary storage block of the storage system, and updating the primary storage block metadata; and (b) reading the data unit from the primary storage block and sending the data unit to a requesting entity.

Abstract: A method for low-latency data mirroring in a storage system, the method may include receiving, by a compute node of the storage system, a read request for reading a data unit stored at a primary storage block of the storage system; determining, by the compute node and based on primary storage block metadata, whether a content of the primary storage block is consistent; and responding to the determining of whether the content of the primary storage block is consistent. The responding may include (a) when determining that the primary storage block is inconsistent—copying the data unit from the primary storage block to a secondary storage block of the storage system, and updating the primary storage block metadata; and (b) reading the data unit from the primary storage block and sending the data unit to a requesting entity.

Abstract: A system and method for replicating a file system. The method includes: copying a portion of the file system from a first storage, wherein the at least a portion of the file system includes underlying data and metadata, wherein the metadata includes pointers to the underlying data and metadata defining a file system hierarchy; partitioning the copied data of the file system into a plurality of blobs, wherein the plurality of blobs includes a plurality of data blobs and a plurality of metadata blobs, wherein each data blob includes at least one portion of the underlying data; generating a plurality of filter objects based on the copied data, wherein each filter object includes a list of metadata blobs; and storing the plurality of blobs and the plurality of filter objects in a second storage.

Abstract: A method is disclosed for destaging data to a storage device set that is arranged to maintain M replicas of the data, the storage device set having M primary storage devices and N secondary storage devices, the method comprising: detecting a destage event; and in response to the destage event, destaging the data item that is stored in a journal, the destaging including: issuing M primary write requests for storing the data item, each of the M primary write requests being directed to a different one of the M primary storage devices; in response to detecting that L of the primary write requests have failed, issuing L secondary write requests for storing the data item, each of the L secondary write requests being directed to a different secondary storage device; updating a bitmap to identify all primary and secondary storage devices where the data item has been stored.

Abstract: A method is disclosed for destaging data to a storage device set that is arranged to maintain M replicas of the data, the storage device set having M primary storage devices and N secondary storage devices, the method comprising: detecting a destage event; and in response to the destage event, destaging the data item that is stored in a journal, the destaging including: issuing M primary write requests for storing the data item, each of the M primary write requests being directed to a different one of the M primary storage devices; in response to detecting that L of the primary write requests have failed, issuing L secondary write requests for storing the data item, each of the L secondary write requests being directed to a different secondary storage device; updating a bitmap to identify all primary and secondary storage devices where the data item has been stored.

Abstract: A system and method for replicating a file system. The method includes: copying a portion of the file system from a first storage, wherein the at least a portion of the file system includes underlying data and metadata, wherein the metadata includes pointers to the underlying data and metadata defining a file system hierarchy; partitioning the copied data of the file system into a plurality of blobs, wherein the plurality of blobs includes a plurality of data blobs and a plurality of metadata blobs, wherein each data blob includes at least one portion of the underlying data; generating a plurality of filter objects based on the copied data, wherein each filter object includes a list of metadata blobs; and storing the plurality of blobs and the plurality of filter objects in a second storage.

Abstract: A clustered storage system in one embodiment comprises a plurality of nodes, with each of at least a subset of the nodes comprising a set of processing modules configured to communicate over one or more networks with corresponding sets of processing modules on other ones of the nodes. In conjunction with a failure of a first instance of a process running on a given one of the nodes and a subsequent restart of a second instance of the process, at least one of the processing modules is to identify at least one transfer buffer command of the first instance of the process, to identify a plurality of logically ordered commands of the first instance of the process, and to provide distinct treatment of the transfer buffer command relative to treatment of the logically ordered commands in a manner that ensures that the restart of the second instance of the process is not delayed to await completion of the transfer buffer command or the logically ordered commands.

Abstract: A clustered storage system in one embodiment comprises a plurality of nodes, with each of at least a subset of the nodes comprising a set of processing modules configured to communicate over one or more networks with corresponding sets of processing modules on other ones of the nodes. In conjunction with a failure of a first instance of a process running on a given one of the nodes and a subsequent restart of a second instance of the process, at least one of the processing modules is to identify at least one transfer buffer command of the first instance of the process, to identify a plurality of logically ordered commands of the first instance of the process, and to provide distinct treatment of the transfer buffer command relative to treatment of the logically ordered commands in a manner that ensures that the restart of the second instance of the process is not delayed to await completion of the transfer buffer command or the logically ordered commands.