Abstract: A method for responding to access requests, the method may include (a) detecting, by a controller of a local storage system (LSS), a read pattern that (i) is associated with a requestor that is in communication with the LSS, and (ii) is estimated to comprise future read requests that are aimed to a remote part of a file system entity (FSE) that is stored at a remote storage system (RSS); wherein there is a latency difference between a remote latency associated with the remote part of the FSE and a LSS latency that is associated with the requestor; and (b) performing a prefetch process of remote sub-parts of the remote part of the FSE in order to support the read pattern while maintaining a desired latency.

Abstract: A method for compressing an evaluated chunk (EC), the method includes (i) selecting a compressing scheme (SC) of a set of candidate SCs to provide a selected SC; and (ii) compressing the EC using the selected SC to provide a compressed EC. A similar chunk that is similar to the EC is tied to a reference chunk (RC) by having the similar chunk compressed based on the RC. The selecting includes evaluating candidate SCs of the set. The selecting is responsive to a first relationship between (i) a similarity score indicative of a similarity between the EV chunk and the similar chunk, and (ii) a similarity score indicative of a similarity between the similar chunk and the RC. A first candidate SC of the set includes untying the similar chunk from the RC, and tying the EV to the similar chunk. A second candidate SC of the set includes self-compressing the EV without tying the EV to the similar chunk.

Abstract: A method for striping, the method may include performing, for each period of time out of multiple periods of time, the steps of: (i) determining striping rules; wherein the determining of the striping rules may include selecting one or more selected stripe size values out of multiple stripe size value candidates; wherein the selecting is based on values of storage system parameters that are obtained when applying the multiple stripe size value candidates; wherein the storage system parameters comprise storage space utilization and storage system throughput; and (ii) applying the striping rules by the storage system, during the period of time; wherein the applying comprises obtaining data chunks; converting the data chunks to stripes having at least one of the one or more selected stripe size values; and storing the stripes in the storage system.

Abstract: A method for selectively untying at least one of compression related links to a stale reference chunk, the method may include determining whether a compression effectiveness condition is fulfilled in relation to a set of referring chunks that are compressed using the stale reference chunk and are linked to the to the stale reference chunk by the compression related links; wherein the compression effectiveness condition is responsive to, at least, a number of the referring chunks of the set; and untying the at least one compression related links to the stale reference chunks when the compression effectiveness condition is not fulfilled.

Abstract: A method for recovering failed chunks, the method may include (i) obtaining a failure indication about a failure of a first number (X1) of failed chunks; wherein the chunks were stored in a group of disks, the group of disks is configured to store a second number (?) stripes of multiple (N) chunks, wherein each stripe comprises a first plurality (K) of data chunks and a second plurality (R) of parity chunks; wherein R exceeds ( X ? 1 ? ) ; and (ii) performing at least one recovery iteration until fulfilling a stop condition. Each recovery iteration of the at least one recovery iteration may include (a.i) selecting valid chunks to provide selected valid chunks, wherein a number of selected chunks is smaller than a product of a multiplication of R by ?; (a.ii) retrieving valid data chunks that are relevant to the selected valid chunks; and (a.iii) attempting to reconstruct the failed chunks based on the retrieved chunks.

Abstract: A method for selectively untying at least one of compression related links to a stale reference chunk, the method may include determining whether a compression effectiveness condition is fulfilled in relation to a set of referring chunks that are compressed using the stale reference chunk and are linked to the to the stale reference chunk by the compression related links; wherein the compression effectiveness condition is responsive to, at least, a number of the referring chunks of the set; and untying the at least one compression related links to the stale reference chunks when the compression effectiveness condition is not fulfilled.

Abstract: A method for striping based on evaluated rules, the method may include determining a compatibility, with a storage system utilization policy, of storing stripes under evaluated rules; wherein the evaluated rules define a stripe size, a number of parity chunks per stripe, and maximal numbers of chunks within a stripe per different failure domains of different size ranges; checking whether the storing of the stripes is compatible with the storage system utilization policy; when finding that the storing of the stripes is not compatible then searching for one or more changes of one or more of the maximal numbers that yields compliant one or more maximal numbers that once applied results in a compliance with the storage system utilization policy; applying the compliant one or more maximal numbers when finding the compliant one or more maximal numbers; and determining that the evaluated failure domain rules are non-compliant when failing to find the compliant one or more maximal numbers.

Abstract: A method for recovering failed chunks, the method may include (i) obtaining a failure indication about a failure of a first number (X1) of failed chunks; wherein the chunks were stored in a group of disks, the group of disks is configured to store a second number (?) stripes of multiple (N) chunks, wherein each stripe comprises a first plurality (K) of data chunks and a second plurality (R) of parity chunks; wherein R exceeds ( X ? 1 ? ) ; and (ii) performing at least one recovery iteration until fulfilling a stop condition. Each recovery iteration of the at least one recovery iteration may include (a.i) selecting valid chunks to provide selected valid chunks, wherein a number of selected chunks is smaller than a product of a multiplication of R by ?; (a.ii) retrieving valid data chunks that are relevant to the selected valid chunks; and (a.iii) attempting to reconstruct the failed chunks based on the retrieved chunks.

Abstract: A method for striping, the method may include performing, for each period of time out of multiple periods of time, the steps of: (i) determining striping rules; wherein the determining of the striping rules may include selecting one or more selected stripe size values out of multiple stripe size value candidates; wherein the selecting is based on values of storage system parameters that are obtained when applying the multiple stripe size value candidates; wherein the storage system parameters comprise storage space utilization and storage system throughput; and (ii) applying the striping rules by the storage system, during the period of time; wherein the applying comprises obtaining data chunks; converting the data chunks to stripes having at least one of the one or more selected stripe size values; and storing the stripes in the storage system.

Abstract: A system and method for erasure coding. The method includes distributing a plurality of data chunks according to a mirroring scheme, wherein the plurality of data chunks is distributed as a plurality of rows among a plurality of non-volatile memory (NVM) nodes, wherein the mirroring scheme defines a plurality of groups, each group including a subset of the plurality of data chunks, wherein each data chunk in a group has a role corresponding to a relative position of the data chunk within the group, wherein data chunks included in the plurality of groups having the same relative positions within their respective groups have the same role, wherein each row of the plurality of rows includes at least one summation data chunk that is a function of at least one data chunk included in the row and of at least one extra data chunk included in at least one other row.

Abstract: A distributed storage system and a method for providing resiliency in distributed storage systems. The distributed storage system includes a plurality of storage nodes including a plurality of disks, wherein the plurality of disks includes a plurality of blocks, wherein the plurality of disks is logically segmented into the plurality of stripes, wherein each of the plurality of stripes is dynamically allocated to a portion of the plurality of blocks distributed across a subset of the plurality of disks; and a plurality of compute nodes, wherein each of the plurality of compute nodes is configured to read data from each of the plurality of storage nodes, wherein each of the plurality of compute nodes is assigned at least one stripe of the plurality of stripes, wherein each of the plurality of compute nodes is configured to write data to each stripe assigned to the compute node.

Abstract: A system and method for erasure coding. The method includes distributing a plurality of data chunks according to a mirroring scheme, wherein the plurality of data chunks is distributed as a plurality of rows among a plurality of non-volatile memory (NVM) nodes, wherein the mirroring scheme defines a plurality of groups, each group including a subset of the plurality of data chunks, wherein each data chunk in a group has a role corresponding to a relative position of the data chunk within the group, wherein data chunks included in the plurality of groups having the same relative positions within their respective groups have the same role, wherein each row of the plurality of rows includes at least one summation data chunk that is a function of at least one data chunk included in the row and of at least one extra data chunk included in at least one other row.

Abstract: A distributed storage system and a method for providing resiliency in distributed storage systems. The distributed storage system includes a plurality of storage nodes including a plurality of disks, wherein the plurality of disks includes a plurality of blocks, wherein the plurality of disks is logically segmented into the plurality of stripes, wherein each of the plurality of stripes is dynamically allocated to a portion of the plurality of blocks distributed across a subset of the plurality of disks; and a plurality of compute nodes, wherein each of the plurality of compute nodes is configured to read data from each of the plurality of storage nodes, wherein each of the plurality of compute nodes is assigned at least one stripe of the plurality of stripes, wherein each of the plurality of compute nodes is configured to write data to each stripe assigned to the compute node.

Abstract: A plurality of server processor workers is configured for the distributed parallel processing of deduplicated data entities in a plurality of chunks. The deduplicated data processing rate is regulated using a rate control mechanism. The rate control mechanism incorporates a debt/credit algorithm specifying which of the plurality of workers processing the deduplicated data entities must wait for each of a plurality of calculated required sleep times, the calculated required sleep times being calculated as a best fit between a maximum allowable sleep time, a delta vector, and a limit per time vector, wherein the required sleep time is one of the plurality of calculated required sleep times.

Abstract: A plurality of workers is configured for parallel processing of deduplicated data entities in a plurality of chunks. The deduplicated data processing rate is regulated using a rate control mechanism. The rate control mechanism incorporates a debt/credit algorithm specifying which of the plurality of workers processing the deduplicated data entities must wait for each of a plurality of calculated required sleep times, the calculated required sleep times being calculated as a best fit between a maximum allowable sleep time, a delta vector, and a limit per time vector, wherein the required sleep time is one of the plurality of calculated required sleep times.

Abstract: A plurality of workers is configured for parallel processing of deduplicated data entities in a plurality of chunks. The deduplicated data processing rate is regulated using a rate control mechanism. The rate control mechanism incorporates a debt/credit algorithm specifying which of the plurality of workers processing the deduplicated data entities must wait for each of a plurality of calculated required sleep times, the calculated required sleep times being calculated as a best fit between a maximum allowable sleep time, a delta vector, and a limit per time vector, wherein the required sleep time is one of the plurality of calculated required sleep times.

Abstract: A plurality of server processor workers is configured for the distributed parallel processing of deduplicated data entities in a plurality of chunks. The deduplicated data processing rate is regulated using a rate control mechanism. The rate control mechanism incorporates a debt/credit algorithm specifying which of the plurality of workers processing the deduplicated data entities must wait for each of a plurality of calculated required sleep times, the calculated required sleep times being calculated as a best fit between a maximum allowable sleep time, a delta vector, and a limit per time vector, wherein the required sleep time is one of the plurality of calculated required sleep times.

Abstract: Workers are configured for parallel processing of deduplicated data entities in chunks. The deduplicated data processing rate is regulated using a rate control mechanism. The rate control mechanism incorporates a debt/credit algorithm specifying which of the workers processing the deduplicated data entities must wait for each of a multiplicity of calculated required sleep times.

Abstract: A plurality of workers is configured for parallel processing of deduplicated data entities in a plurality of chunks. The deduplicated data processing rate is regulated using a rate control mechanism. The rate control mechanism incorporates a debt/credit algorithm specifying which of the plurality of workers processing the deduplicated data entities must wait for each of a plurality of calculated required sleep times. The rate control mechanism limits a data flow rate based on a penalty acquired during a last processing of one of the plurality of chunks in a retroactive manner, and operates on at least one vector representation of at least one limit specification to accommodate a variety of available dimensions corresponding to the at least one limit specification.

Abstract: A plurality of workers is configured for parallel processing of deduplicated data entities in a plurality of chunks. The deduplicated data processing rate is regulated using a rate control mechanism. The rate control mechanism incorporates a debt/credit algorithm specifying which of the plurality of workers processing the deduplicated data entities must wait for each of a plurality of calculated required sleep times. The rate control mechanism is adapted to limit a data flow rate based on a penalty acquired during a last processing of one of the plurality of chunks in a retroactive manner, and further adapted to operate on at least one vector representation of at least one limit specification to accommodate a variety of available dimensions corresponding to the at least one limit specification.