Abstract: A system divides non-volatile memory of a plurality of storage devices into physical extents which comprises chunks. The system allocates slabs associated with the storage devices, wherein a respective slab comprises extents from different storage devices and further comprises stripes. A stripe comprises a chunk from each extent of the respective slab. The system updates, in a first data structure, an entry which indicates: a slab number for the respective allocated slab; and a storage device identifier and an extent number for each extent in the respective allocated slab. Responsive to receiving a write request, the system obtains a first stripe from a pre-allocated list which includes the allocated slabs. The system searches, based on stripe information associated with the first stripe, the first data structure to obtain a physical location in a storage device to which to issue the write request.

Abstract: One aspect facilitates a global map in a distributed system. The system generates a first data structure which comprises key-value pairs, wherein, in a respective key-value pair, the respective key is an integer and the respective value comprises a pointer to a sparse array which includes a bitmap (such as an ordered two-bit bitmap). The system stores the first data structure as a first partition of the global map. The system searches, based on a pattern, the first data structure to obtain a first value. If the first value comprises a two-bit bitmap, the system converts, based on the pattern, the first value to a two-dimensional bitmap, and performs a function on the first value to obtain a first result. The system uses the first value or the first result as metadata to execute a read or write operation in a filesystem associated with the distributed system.

Abstract: A system divides non-volatile memory of a plurality of storage devices into physical extents which comprises chunks. The system allocates slabs associated with the storage devices, wherein a respective slab comprises extents from different storage devices and further comprises stripes. A stripe comprises a chunk from each extent of the respective slab. The system updates, in a first data structure, an entry which indicates: a slab number for the respective allocated slab; and a storage device identifier and an extent number for each extent in the respective allocated slab. Responsive to receiving a write request, the system obtains a first stripe from a pre-allocated list which includes the allocated slabs. The system searches, based on stripe information associated with the first stripe, the first data structure to obtain a physical location in a storage device to which to issue the write request.

Abstract: The system allocates, in a distributed system comprising a plurality of nodes, a plurality of portions of memory which comprise shared remote memory content. The system registers the allocated portions with an operating system to be accessed via RDMA. The system accesses, by a first node, the allocated portions to obtain a local copy. The system performs an atomic operation on one or more bits of the shared remote memory content via libfabric atomic application programming interface calls, by one or more of: updating the one or more bits based on a new value and an offset; retrieving, from the shared remote memory content based on the offset, a current value of the one or more bits prior to the updating; and performing an action on the shared remote memory content based on a comparison of the retrieved current value with an expected value in the local copy.

Abstract: One aspect facilitates a global map in a distributed system. The system generates a first data structure which comprises key-value pairs, wherein, in a respective key-value pair, the respective key is an integer and the respective value comprises a pointer to a sparse array which includes a bitmap (such as an ordered two-bit bitmap). The system stores the first data structure as a first partition of the global map. The system searches, based on a pattern, the first data structure to obtain a first value. If the first value comprises a two-bit bitmap, the system converts, based on the pattern, the first value to a two-dimensional bitmap, and performs a function on the first value to obtain a first result. The system uses the first value or the first result as metadata to execute a read or write operation in a filesystem associated with the distributed system.

Abstract: The present technology provides a two step process for providing a linearized dynamic storage pool. First, physical storage devices are abstracted. The physical storage devices used for the pool are divided into extents, grouped by storage class, and stripes are created from data chunks of similar classified devices. A virtual volume is then provisioned from and the virtual volume is divided into virtual stripes. A volume map is created to map the virtual stripes with data to the physical stripes, linearly mapping the virtual layout to the physical capacity to maintain optimal performance.

Abstract: A first snapshot is taken of a first block storage resource that is initially identical in content to a second block storage resource. A second snapshot of the first block storage resource is taken at a later time. A record is kept of all blocks modified on the first block storage resource. Only those blocks modified between the time of the first and second snapshots are written to the second block storage resource. After all the modified blocks are written to the second block storage resource, a snapshot is taken of the second block storage resource to maintain a consistent snapshot of the second block storage resource in case of communication failure during the next round. The first snapshot is then deleted, the second takes the role of the first, and the next round of replication begins.

Abstract: A first snapshot is taken of a first block storage resource that is initially identical in content to a second block storage resource. A second snapshot of the first block storage resource is taken at a later time. A record is kept of all blocks modified on the first block storage resource. Only those blocks modified between the time of the first and second snapshots are written to the second block storage resource. After all the modified blocks are written to the second block storage resource, a snapshot is taken of the second block storage resource to maintain a consistent snapshot of the second block storage resource in case of communication failure during the next round. The first snapshot is then deleted, the second takes the role of the first, and the next round of replication begins.