Abstract: A method for storage system data aware compression, the method may include pre-compressing data units received by the storage system, by different pre-compression units to provide different pre-compressed versions of the data units; wherein the different pre-compression schemes are associated with different compression schemes, wherein at least some of the different compression schemes are data type specific compression schemes; calculating entropies of the different pre-compressed versions; and selecting a compression scheme out of the different compression schemes based on the entropies of the different pre-compressed versions.

Abstract: A system and method for using free space for recovering erasure coding data sets. The method includes segmenting at least one erasure coding data set into at least one stripe based on an erasure coding scheme, wherein each erasure coding data set includes chunks, wherein the chunks include chunks of systematic data and chunks of parity data; distributing the at least one stripe across non-volatile memory nodes based on the erasure coding scheme, wherein each non-volatile memory node is a unit of a non-volatile storage device; and upon a first failure of one of the non-volatile memory nodes: detecting that at least one of the non-volatile memory nodes that is not failed includes a first free space that does not currently store user data; and performing a first recovery process for recovering a first portion of the at least one erasure coding data set, based on the first free space.

Abstract: A method for striping, the method may include performing, for each period of time out of multiple periods of time, the steps of: (i) determining striping rules; wherein the determining of the striping rules may include selecting one or more selected stripe size values out of multiple stripe size value candidates; wherein the selecting is based on values of storage system parameters that are obtained when applying the multiple stripe size value candidates; wherein the storage system parameters comprise storage space utilization and storage system throughput; and (ii) applying the striping rules by the storage system, during the period of time; wherein the applying comprises obtaining data chunks; converting the data chunks to stripes having at least one of the one or more selected stripe size values; and storing the stripes in the storage system.

Abstract: A method for selectively untying at least one of compression related links to a stale reference chunk, the method may include determining whether a compression effectiveness condition is fulfilled in relation to a set of referring chunks that are compressed using the stale reference chunk and are linked to the to the stale reference chunk by the compression related links; wherein the compression effectiveness condition is responsive to, at least, a number of the referring chunks of the set; and untying the at least one compression related links to the stale reference chunks when the compression effectiveness condition is not fulfilled.

Abstract: A method for storage system data aware compression, the method may include pre-compressing data units received by the storage system, by different pre-compression units to provide different pre-compressed versions of the data units; wherein the different pre-compression schemes are associated with different compression schemes, wherein at least some of the different compression schemes are data type specific compression schemes; calculating entropies of the different pre-compressed versions; and selecting a compression scheme out of the different compression schemes based on the entropies of the different pre-compressed versions.

Abstract: A method for recovering failed chunks, the method may include (i) obtaining a failure indication about a failure of a first number (X1) of failed chunks; wherein the chunks were stored in a group of disks, the group of disks is configured to store a second number (?) stripes of multiple (N) chunks, wherein each stripe comprises a first plurality (K) of data chunks and a second plurality (R) of parity chunks; wherein R exceeds ( X ? 1 ? ) ; and (ii) performing at least one recovery iteration until fulfilling a stop condition. Each recovery iteration of the at least one recovery iteration may include (a.i) selecting valid chunks to provide selected valid chunks, wherein a number of selected chunks is smaller than a product of a multiplication of R by ?; (a.ii) retrieving valid data chunks that are relevant to the selected valid chunks; and (a.iii) attempting to reconstruct the failed chunks based on the retrieved chunks.

Abstract: A method for hierarchical workload allocation in a storage system, the method may include determining to reallocate a compute workload of a current compute core of the storage system; wherein the current compute core is responsible for executing a workload allocation unit that comprises one or more first type shards; and reallocating the compute workload by (a) maintaining the responsibility of the current compute core for executing the workload allocation unit, and (b) reallocating at least one first type shard of the one or more first type shards to a new workload allocation unit that is allocated to a new compute core of new compute cores.

Abstract: A method for selectively untying at least one of compression related links to a stale reference chunk, the method may include determining whether a compression effectiveness condition is fulfilled in relation to a set of referring chunks that are compressed using the stale reference chunk and are linked to the to the stale reference chunk by the compression related links; wherein the compression effectiveness condition is responsive to, at least, a number of the referring chunks of the set; and untying the at least one compression related links to the stale reference chunks when the compression effectiveness condition is not fulfilled.

Abstract: A method for power management of a storage system unit, the method may include selecting a power reduction measure out of (a) a shutdown of at least one stateless compute node out of stateless compute nodes of the storage system unit, and (b) an other power reduction measure that does not involve the shutdown of the at least one stateless compute node, while prioritizing the shutdown of the at least one stateless compute node over the other power reduction measure; wherein the storage system unit further comprises storage nodes; wherein states related to communications with client computers (i) are maintained in the storage nodes, and (ii) are not stored on the stateless compute nodes; and applying the power reduction measure based on the selecting.

Abstract: A method for generating a reconstructed version of a filesystem entity, the method may include (i) generating fallback retrieval metadata for a reconstructed version segment, when the reconstructed version segment is (a) associated with a corresponding intermediate version segment, and (b) the corresponding intermediate version segment is preceded by a corresponding most updated segment that does not exceed a fallback version of the filesystem entity; wherein the reconstructed version segment, the corresponding intermediate version segment and the corresponding most updated segment that does not exceed a fallback version have a same address range; (ii) generating a non-existing indicator for the reconstructed version segment, when the reconstructed version segment is (a) associated with the corresponding intermediate version segment, and (b) the corresponding intermediate version segment is not preceded by any corresponding most updated segment that does not exceed the fallback version; and (iii) maintaining

Abstract: A method for generating a reconstructed version of a filesystem entity, the method may include (i) generating fallback retrieval metadata for a reconstructed version segment, when the reconstructed version segment is (a) associated with a corresponding intermediate version segment, and (b) the corresponding intermediate version segment is preceded by a corresponding most updated segment that does not exceed a fallback version of the filesystem entity; wherein the reconstructed version segment, the corresponding intermediate version segment and the corresponding most updated segment that does not exceed a fallback version have a same address range; (ii) generating a non-existing indicator for the reconstructed version segment, when the reconstructed version segment is (a) associated with the corresponding intermediate version segment, and (b) the corresponding intermediate version segment is not preceded by any corresponding most updated segment that does not exceed the fallback version; and (iii) maintaining

Abstract: A method for striping based on evaluated rules, the method may include determining a compatibility, with a storage system utilization policy, of storing stripes under evaluated rules; wherein the evaluated rules define a stripe size, a number of parity chunks per stripe, and maximal numbers of chunks within a stripe per different failure domains of different size ranges; checking whether the storing of the stripes is compatible with the storage system utilization policy; when finding that the storing of the stripes is not compatible then searching for one or more changes of one or more of the maximal numbers that yields compliant one or more maximal numbers that once applied results in a compliance with the storage system utilization policy; applying the compliant one or more maximal numbers when finding the compliant one or more maximal numbers; and determining that the evaluated failure domain rules are non-compliant when failing to find the compliant one or more maximal numbers.

Abstract: A method for recovering failed chunks, the method may include (i) obtaining a failure indication about a failure of a first number (X1) of failed chunks; wherein the chunks were stored in a group of disks, the group of disks is configured to store a second number (?) stripes of multiple (N) chunks, wherein each stripe comprises a first plurality (K) of data chunks and a second plurality (R) of parity chunks; wherein R exceeds ( X ? 1 ? ) ; and (ii) performing at least one recovery iteration until fulfilling a stop condition. Each recovery iteration of the at least one recovery iteration may include (a.i) selecting valid chunks to provide selected valid chunks, wherein a number of selected chunks is smaller than a product of a multiplication of R by ?; (a.ii) retrieving valid data chunks that are relevant to the selected valid chunks; and (a.iii) attempting to reconstruct the failed chunks based on the retrieved chunks.

Abstract: A method for striping, the method may include performing, for each period of time out of multiple periods of time, the steps of: (i) determining striping rules; wherein the determining of the striping rules may include selecting one or more selected stripe size values out of multiple stripe size value candidates; wherein the selecting is based on values of storage system parameters that are obtained when applying the multiple stripe size value candidates; wherein the storage system parameters comprise storage space utilization and storage system throughput; and (ii) applying the striping rules by the storage system, during the period of time; wherein the applying comprises obtaining data chunks; converting the data chunks to stripes having at least one of the one or more selected stripe size values; and storing the stripes in the storage system.

Abstract: A method for hierarchical workload allocation in a storage system, the method may include determining to reallocate a compute workload of a current compute core of the storage system; wherein the current compute core is responsible for executing a workload allocation unit that comprises one or more first type shards; and reallocating the compute workload by (a) maintaining the responsibility of the current compute core for executing the workload allocation unit, and (b) reallocating at least one first type shard of the one or more first type shards to a new workload allocation unit that is allocated to a new compute core of new compute cores.

Abstract: A system and method for prolonging lifespans of storage drives. The method includes determining an expected expiration time for each of a plurality of blocks, wherein each block includes data of a respective file, wherein the expected expiration of each block is determined based on a file type of the respective file; and writing a portion of data to at least one block of the plurality of blocks based on the expected expiration time for each block.

Abstract: A method for hierarchical workload allocation in a storage system, the method may include determining to reallocate a compute workload of a current compute core of the storage system; wherein the current compute core is responsible for executing a workload allocation unit that comprises one or more first type shards; and reallocating the compute workload by (a) maintaining the responsibility of the current compute core for executing the workload allocation unit, and (b) reallocating at least one first type shard of the one or more first type shards to a new workload allocation unit that is allocated to a new compute core of new compute cores.

Abstract: A system and method for time synchronization. The method includes incrementing at least one instance of a snap value, wherein the at least one instance is stored in at least one storage node of a distributed storage system, wherein each of the at least one instance is incremented using a single-phase commit operation, wherein each of the plurality of storage nodes includes at least one timestamp, wherein each timestamp indicates a corresponding snap time, wherein each snap time is determined by reading one of the at least one instance at a time of creation of the corresponding timestamp.

Abstract: A system and method for using free space for recovering erasure coding data sets. The method includes segmenting at least one erasure coding data set into at least one stripe based on an erasure coding scheme, wherein each erasure coding data set includes chunks, wherein the chunks include chunks of systematic data and chunks of parity data; distributing the at least one stripe across non-volatile memory nodes based on the erasure coding scheme, wherein each non-volatile memory node is a unit of a non-volatile storage device; and upon a first failure of one of the non-volatile memory nodes: detecting that at least one of the non-volatile memory nodes that is not failed includes a first free space that does not currently store user data; and performing a first recovery process for recovering a first portion of the at least one erasure coding data set, based on the first free space.

Abstract: A system and method for erasure coding. The method includes distributing a plurality of data chunks according to a mirroring scheme, wherein the plurality of data chunks is distributed as a plurality of rows among a plurality of non-volatile memory (NVM) nodes, wherein the mirroring scheme defines a plurality of groups, each group including a subset of the plurality of data chunks, wherein each data chunk in a group has a role corresponding to a relative position of the data chunk within the group, wherein data chunks included in the plurality of groups having the same relative positions within their respective groups have the same role, wherein each row of the plurality of rows includes at least one summation data chunk that is a function of at least one data chunk included in the row and of at least one extra data chunk included in at least one other row.