Abstract: Systems and methods for scalable locking. A method includes adding a first lock entry representing a pending lock to a first tree, the first lock entry indicating a range to be locked; checking at least a portion of at least one second tree to determine whether a conflicting lock exists for the first lock entry among at least one second lock entry based on the range to be locked, wherein each of the first tree and the at least one second tree is a data structure including a plurality of nodes representing at least a plurality of attributes, wherein the plurality of attributes of the at least one second tree includes the at least one second lock entry; committing the pending lock when no conflicting lock exists; and resolving the pending lock based on a resolution of the conflicting lock when a conflicting lock exists.

Abstract: A system and method for transaction management. The system includes a processing circuitry; and a memory, the memory containing instructions that, when executed by the processing circuitry, configure the system to: assign a transaction identifier (ID) to a transaction, wherein the transaction includes at least one modification to at least one object stored in a storage system, each object having a persistent state, wherein the transaction ID indicates a state of the transaction, wherein the state of the transaction initially indicates that the transaction is in progress; modify the at least one object, wherein the modifying further comprises adding at least one conditional entry to each object, wherein each conditional entry includes one of the at least one modification that is conditional on the state of the transaction; and update the transaction ID when a termination event occurs, wherein the updating further includes changing the state of the transaction.

Abstract: A distributed storage system. The distributed storage system includes a plurality of storage nodes (DNodes), wherein the DNodes are configured to store a plurality of elements and a plurality of data blocks, wherein each element is a persistent metadata structure, wherein at least one of the elements store at least one attribute, wherein the at least one attribute includes a plurality of pointers; and a plurality of compute nodes (CNodes), wherein each CNode has access to each of the DNodes, wherein each CNode is assigned to a respective at least one cluster of a plurality of clusters, wherein each CNode is configured to receive at least a portion of access commands directed to the respective at least one cluster and to execute the received access commands based on the plurality of elements.

Abstract: A system and method for efficiently providing snapshots in storage systems. The method includes creating a plurality of snaplines for a storage system, each snapline having metadata including a timestamp and a status marker for each of a plurality of offset ranges, wherein each offset range corresponds to data stored in the storage system, the storage system including a plurality of compute nodes, wherein the timestamp of each snapline is a value of a global counter at a time of creation of the snapline, wherein the global counter is uniform among the plurality of compute nodes; and creating a snapshot based on the plurality of snaplines and a time value, wherein the snapshot indicates a snapline of the plurality of snaplines that is not to be deleted, wherein the timestamp of the snapline that is not to be deleted corresponds to the time value.

Abstract: Systems and methods for leader election. A disclosed method includes sending, by a first compute node of a plurality of compute nodes, a plurality of remote procedure calls (RPCs) to a plurality of storage boxes according to an order, wherein each of the plurality of RPCs causes a leader election algorithm to execute in one of the plurality of storage boxes; and updating a state of the first compute node to “leader” when a result of executing the leader election algorithm for each of the plurality of RPCs indicates that the first compute node is elected as a leader node.

Abstract: Systems and methods for leader election. A disclosed method includes sending, by a first compute node of a plurality of compute nodes, a plurality of remote procedure calls (RPCs) to a plurality of storage boxes according to an order, wherein each of the plurality of RPCs causes a leader election algorithm to execute in one of the plurality of storage boxes; and updating a state of the first compute node to “leader” when a result of executing the leader election algorithm for each of the plurality of RPCs indicates that the first compute node is elected as a leader node.

Abstract: A system and method for determining ancestry in directed acyclic graphs (DAGs). The method includes accessing a first node of a plurality of nodes in a DAG to obtain first change time data and ancestry data, wherein the plurality of nodes includes at least one point of interest, wherein the first change time data indicates a time of a first change to ancestry of the closest point of interest (CPOI); accessing a CPOI of the first node to obtain second change time data, wherein the second change time data indicates a time of a most recent change to ancestry of the CPOI; comparing the first change time data with the second change time data; and determining ancestry of the first node, wherein the ancestry of the first node is determined based on the ancestry data when the first change time data matches the second change time data.

Abstract: A distributed storage system and a method for providing resiliency in distributed storage systems. The distributed storage system includes a plurality of storage nodes including a plurality of disks, wherein the plurality of disks includes a plurality of blocks, wherein the plurality of disks is logically segmented into the plurality of stripes, wherein each of the plurality of stripes is dynamically allocated to a portion of the plurality of blocks distributed across a subset of the plurality of disks; and a plurality of compute nodes, wherein each of the plurality of compute nodes is configured to read data from each of the plurality of storage nodes, wherein each of the plurality of compute nodes is assigned at least one stripe of the plurality of stripes, wherein each of the plurality of compute nodes is configured to write data to each stripe assigned to the compute node.

Abstract: A system and method for prolonging lifespans of storage drives. The method includes determining an expected expiration time for each of a plurality of blocks, wherein each block includes data of a respective file, wherein the expected expiration of each block is determined based on a file type of the respective file; and writing a portion of data to at least one block of the plurality of blocks based on the expected expiration time for each block.

Abstract: A system and method for time synchronization. The method includes incrementing at least one instance of a snap value, wherein the at least one instance is stored in at least one storage node of a distributed storage system, wherein each of the at least one instance is incremented using a single-phase commit operation, wherein each of the plurality of storage nodes includes at least one timestamp, wherein each timestamp indicates a corresponding snap time, wherein each snap time is determined by reading one of the at least one instance at a time of creation of the corresponding timestamp.

Abstract: A system and method for replicating a file system. The method includes: copying a portion of the file system from a first storage, wherein the at least a portion of the file system includes underlying data and metadata, wherein the metadata includes pointers to the underlying data and metadata defining a file system hierarchy; partitioning the copied data of the file system into a plurality of blobs, wherein the plurality of blobs includes a plurality of data blobs and a plurality of metadata blobs, wherein each data blob includes at least one portion of the underlying data; generating a plurality of filter objects based on the copied data, wherein each filter object includes a list of metadata blobs; and storing the plurality of blobs and the plurality of filter objects in a second storage.

Abstract: A system and method for erasure coding. The method includes distributing a plurality of data chunks according to a mirroring scheme, wherein the plurality of data chunks is distributed as a plurality of rows among a plurality of non-volatile memory (NVM) nodes, wherein the mirroring scheme defines a plurality of groups, each group including a subset of the plurality of data chunks, wherein each data chunk in a group has a role corresponding to a relative position of the data chunk within the group, wherein data chunks included in the plurality of groups having the same relative positions within their respective groups have the same role, wherein each row of the plurality of rows includes at least one summation data chunk that is a function of at least one data chunk included in the row and of at least one extra data chunk included in at least one other row.

Abstract: A storage system is provided. The storage system includes a backplane; a plurality of single port storage devices; and a plurality of controllers, wherein the backplane routes a plurality of interconnection lanes between the plurality of controllers and the plurality of single port storage devices, wherein the plurality of controllers is configured to: enable at least one second interconnection lane of the plurality of interconnection lanes when a first controller of the plurality of controllers has failed, wherein a first interconnection lane of the plurality of interconnection lanes is between the first controller and a first single port storage device of the plurality of single port storage devices, wherein the at least one second interconnection lane is between a second controller of the plurality of controllers and the first single port storage device.

Abstract: A system and method for prolonging lifespans of storage drives. The method includes writing at least a first portion of data to at least one high endurance drive; and writing at least a second portion of data to at least one low endurance drive, wherein a storage system includes a plurality of drives, wherein the plurality of drives includes the at least one high endurance drive and the at least one low endurance drive, wherein each high endurance drive has a longer total lifespan than each low endurance drive.

Abstract: A system and method for using free space to improve erasure code locality. The method includes logically segmenting an erasure coding data set into a stripe based on an erasure coding scheme, wherein the erasure coding data set includes a plurality of chunks, wherein the plurality of chunks includes a plurality of chunks of systematic data and a plurality of chunks of parity data, wherein the stripe includes free user data; and distributing the stripe across a plurality of non-volatile memory nodes based on the erasure coding scheme, wherein the free user data is stored in at least one memory location among the plurality of non-volatile memory nodes, wherein each non-volatile node is a unit of non-volatile memory.

Abstract: A distributed storage system and a method for providing resiliency in distributed storage systems. The distributed storage system includes a plurality of storage nodes including a plurality of disks, wherein the plurality of disks includes a plurality of blocks, wherein the plurality of disks is logically segmented into the plurality of stripes, wherein each of the plurality of stripes is dynamically allocated to a portion of the plurality of blocks distributed across a subset of the plurality of disks; and a plurality of compute nodes, wherein each of the plurality of compute nodes is configured to read data from each of the plurality of storage nodes, wherein each of the plurality of compute nodes is assigned at least one stripe of the plurality of stripes, wherein each of the plurality of compute nodes is configured to write data to each stripe assigned to the compute node.

Abstract: A storage system. The storage system includes a plurality of storage nodes (DNodes), wherein the DNodes are configured to store a plurality of elements and a plurality of data blocks, wherein each element is a persistent metadata structure, wherein at least one of the elements store at least one attribute, wherein the at least one attribute includes a plurality of pointers, wherein at least one of the data blocks is stored in at least one of the elements when the at least one of the data blocks has a size below a threshold size; and a plurality of compute nodes (CNodes), wherein each CNode has access to each of the DNodes, wherein each CNode is configured to receive an access command and to execute the access command based on the elements.

Abstract: A storage system. The storage system includes a data store including a plurality of data blocks; and an element store, the element store including a plurality of element trees, wherein each element tree includes a plurality of metadata blocks containing pointers, wherein data of the plurality of data blocks is read by navigating at least one of the plurality of element trees, wherein the element store includes a first level and at least one second level, wherein the first level is shared among the plurality of plurality of element trees.

Abstract: A system and method for efficiently providing snapshots in storage systems. The method includes creating a plurality of snaplines for a storage system, each snapline having metadata including a timestamp and a status marker for each of a plurality of offset ranges, wherein each offset range corresponds to data stored in the storage system, the storage system including a plurality of compute nodes, wherein the timestamp of each snapline is a value of a global counter at a time of creation of the snapline, wherein the global counter is uniform among the plurality of compute nodes; and creating a snapshot based on the plurality of snaplines and a time value, wherein the snapshot indicates a snapline of the plurality of snaplines that is not to be deleted, wherein the timestamp of the snapline that is not to be deleted corresponds to the time value.

Abstract: A storage system. The storage system includes a plurality of storage nodes (DNodes), wherein the DNodes are configured to store a plurality of elements and a plurality of data blocks, wherein each element is a persistent metadata structure, wherein at least one of the elements store at least one attribute, wherein the at least one attribute includes a plurality of pointers; and a plurality of compute nodes (CNodes), wherein each CNode has access to each of the DNodes, wherein each CNode is configured to receive an access command and to execute the access command based on the elements.