Abstract: The present disclosure provides a method for performing sampling operations within a key-value storage engine for AI training workflows, comprising organizing data as key-value pairs within the key-value storage engine, where each key is stored in memory and points to a corresponding value stored in a storage unit, implementing an enhanced iterator initialization function that accepts a database name, a start key, a sampling ratio parameter that determines a proportion of data to be scanned from an entire database, and a seed parameter that serves as a randomization seed, executing a random permutation over a subset of the dataset based on the sampling ratio and seed parameters, and returning values based on the randomized permutation using iterator operations, thereby performing sampling operations directly within the key-value storage engine without requiring intermediate data transfers.

Abstract: A method for updating a key-value (KV) storage system, the method includes (i) receiving a request to write a KV key to a row cache of the KV storage system, wherein the request is associated with a row cache key (RCK), wherein the RCK comprises a row cache identifier (RCID) and a column family identifier (CFID), wherein the KV storage system further comprises a separated database, the separated database comprises a log structured merge (LSM) tree and KV database that is separated from LSM tree; and (ii) invalidating, responsive to the reception of the request, any KV key already cached in the row cache and associated with a same RCK as the RCK associated with the request.

Abstract: A method for updating a key-value (KV) storage system, the method includes (a) receiving a request to write a KV pair to the KV storage system; (b) storing the KV pair in an input buffer that is maintained in a volatile memory module of the KV storage system; (c) storing the KV pair in a separated KV database that is maintained in a non-volatile memory module of the KV storage system; (d) updating a B-tree that is stored in the non-volatile memory unit with (i) a key of the KV pair, (ii) a KV pair timestamp, and (iii) retrieval information for retrieving the KV pair from the separated KV database; and (e) maintaining a backup B-tree in the non-volatile memory module of the KV storage system, wherein the B-tree and the backup B-tree are separated from the separated KV database.

Abstract: A method for performing a backup in relation to a separated database, the method includes creating, by a processing circuit, a new backup log structured merge (LSM) tree of the separated database, wherein the new backup LSM tree is associated with a new point in time (PIT) and belongs to a group of LSM trees that comprise a primary LSM tree and one or more backup LSM trees that are associated with different PITs, and share a key value (KV) database that is mutable and is separated from the group; wherein the creating comprises storing the new backup LSM tree to a destination.

Abstract: A method for managing a log structured merged (LSM) tree of key value (KV) pairs, the LSM tree is stored in a non-volatile memory, the method may include writing a current run from a buffer to a current run location within the LSM tree, the current run comprises current KV pairs; generating or receiving current fingerprints that are indicative of the current KV pairs; performing a run writing update of a management data structure (MDS) by adding to the MDS, mappings between the current KV pairs, the current fingerprints and a current run identifier; updating the LSM tree by merging at least some runs of the LSM tree; and performing a merge update of the MDS to represent the merging.

Abstract: A method for inserting a KV pair to a separated database, the method may include receiving a request to insert the KV pair to the separated database, wherein the separated database comprises a log structured merge (LSM) tree and KV database that is separated from LSM tree; determining whether the KV pair should be associated with a versioned LSM entry or with a non-versions LSM entry; and inserting the KV pair and a KV timestamp in the separated database according to the determining; wherein the inserting includes: storing a combination of the value and the KV timestamp in the KV database; defining an access key to the KV database; wherein the access key is based on the combination when determining that the KV pair should be associated with a versioned LSM; and wherein the access key is based on the key and not on the timestamp when determining that the KV pair should be associated with a non-versioned LSM.

Abstract: A method for managing multiple checkpoints stored in a memory unit, the method may include (a) managing a checkpoint tree, by a memory controller; the checkpoint tree comprises reference nodes and active nodes; wherein a reference node of the reference nodes holds a snapshot of a parent checkpoint taken at the time that a child checkpoint was created from the parent checkpoint; wherein an active node of the active nodes that is associated with the parent node stores changes to the parent checkpoint introduced after a creation of the active node; and wherein the parent checkpoint and the child checkpoint belong to multiple checkpoints stored in the checkpoint tree; (b) receiving access requests to access key value (KV) pairs of one or more checkpoints of the multiple checkpoints; and (c) responding to the access requests based, at least in part, on the checkpoint tree.

Abstract: A method for error correction of logical pages of an erase block of a solid state drive (SSD) memory, the method may include determining an erase block score of the erase block, wherein the calculating is based on a program erase (PE) cycle of the erase block and one or more erase block error correction parameter; determining, based on (a) the erase block score, and (b) a mapping between the erase block score and one or more page error correction parameters for each page type out of multiple pages types, the one or more page error correction parameter for each page type; and allocating, within each page of the erase block, an overprovisioning space and an error correction space, based on at least one page error correction parameter related to a page type of the page.

Abstract: A method for managing a log structured merged (LSM) tree of key value (KV) pairs, the LSM tree is stored in a non-volatile memory, the method may include merging runs of the LSM tree to provide merged runs; writing merged runs to the non-volatile memory; adding new runs to the LSM tree, wherein the adding comprises writing runs to the non-volatile memory; and updating at least one management data structure (MDS) to reflect the merging and the adding; wherein an MDS of the at least one MDS stores a mapping between keys of the KV pairs of the LSM tree, fingerprints associated with the KV pairs of the LSM tree, and compressed run identifiers that identify runs of the LSM tree.

Abstract: A method for performing a backup in relation to a separated database, the method includes creating, by a processing circuit, a new backup log structured merge (LSM) tree of the separated database, wherein the new backup LSM tree is associated with a new point in time (PIT) and belongs to a group of LSM trees that comprise a primary LSM tree and one or more backup LSM trees that are associated with different PITs, and share a key value (KV) database that is mutable and is separated from the group; wherein the creating comprises storing the new backup LSM tree to a destination.

Abstract: There may be provided, systems, method and non-transitory computer readable media for accessing a key value pair stored in a solid state drive (SSD) memory, the method may include calculating, by a SSD memory controller and based on an input key, a first bucket identifier and a first inter-bucket value; determining a block cluster that stores the key pair value, based on the first bucket identifier, the first inter-bucket value and first metadata of a first data structure; calculating, based on the input key, a second bucket identifier and a second inter-bucket value; determining at least one of a key pair value retrieval information and a representative key pair value retrieval information, based on the second bucket identifier, the second inter-bucket value and second metadata of a second data structure; wherein the second data structure is allocated to the block cluster; wherein the second metadata comprises second logical slots, a second slot locator and second collision separation metadata; wherein at le

Abstract: A method for updating a log structured merged (LSM) tree, the method includes (a) performing preemptive full merge operations at first LSM tree levels; and (b) performing capacity triggered merge operations at second LSM tree levels while imposing one or more restrictions; wherein the second LSM tree levels comprise a largest LSM tree level and one or more other second LSM tree levels that are larger from each first LSM tree level; wherein files of the one or more other second LSM tree levels are aligned with files of the largest LSM tree level.

Abstract: A method for accessing a block of information stored in a SSD memory, the method may include obtaining, by an SSD controller, an identifier associated with the block of information; accessing, using the identifier, a first data structure that maps identifiers to cluster maps; wherein the first data structure comprises block sequence metadata, wherein for at least one cluster map, the block sequence metadata comprises a sequence identifier of a sequence of blocks of information that are sequentially written to the SSD memory and are stored together in a cluster of the SSD memory; accessing a cluster map of a cluster that is associated with the sequence provide block retrieval information; and retrieving the block of information from the SSD memory, using the block retrieval information.

Abstract: A method for error correction of logical pages of an erase block of a solid state drive (SSD) memory, the method may include determining an erase block score of the erase block, wherein the calculating is based on a program erase (PE) cycle of the erase block and one or more erase block error correction parameter; determining, based on (a) the erase block score, and (b) a mapping between the erase block score and one or more page error correction parameters for each page type out of multiple pages types, the one or more page error correction parameter for each page type; and allocating, within each page of the erase block, an overprovisioning space and an error correction space, based on at least one page error correction parameter related to a page type of the page.

Abstract: A method for protecting objects, the method may include receiving, by a storage controller, a request to write a new object to a storage-system that may include multiple nodes, each node may include at least one storage element; the new object may include a key value pair; writing the new object to a target node and to another node of the multiple nodes; and generating a new stripe that may include multiple sets of objects, different sets of objects are stored at different nodes of the multiple nodes, a certain set of objects of the multiple sets of objects may include the new object and is stored in a first node of the target node and the other node, any set of objects of the multiple sets of objects is reconstructible using other sets of objects of the multiple set of objects.

Abstract: A method for managing a log structured merged (LSM) tree of key value (KV) pairs, the LSM tree is stored in a non-volatile memory, the method may include writing a current run from a buffer to a current run location within the LSM tree, the current run comprises current KV pairs; generating or receiving current fingerprints that are indicative of the current KV pairs; performing a run writing update of a management data structure (MDS) by adding to the MDS, mappings between the current KV pairs, the current fingerprints and a current run identifier; updating the LSM tree by merging at least some runs of the LSM tree; and performing a merge update of the MDS to represent the merging.

Abstract: A method for managing a log structured merged (LSM) tree of key value (KV) pairs, the LSM tree is stored in a non-volatile memory, the method may include merging runs of the LSM tree to provide merged runs; writing merged runs to the non-volatile memory; adding new runs to the LSM tree, wherein the adding comprises writing runs to the non-volatile memory; and updating at least one management data structure (MDS) to reflect the merging and the adding; wherein an MDS of the at least one MDS stores a mapping between keys of the KV pairs of the LSM tree, fingerprints associated with the KV pairs of the LSM tree, and compressed run identifiers that identify runs of the LSM tree

Abstract: A method for error correction of logical pages of an erase block of a solid state drive (SSD) memory, the method may include determining an erase block score of the erase block, wherein the calculating is based on a program erase (PE) cycle of the erase block and one or more erase block error correction parameter; determining, based on (a) the erase block score, and (b) a mapping between the erase block score and one or more page error correction parameters for each page type out of multiple pages types, the one or more page error correction parameter for each page type; and allocating, within each page of the erase block, an overprovisioning space and an error correction space, based on at least one page error correction parameter related to a page type of the page.

Abstract: There may be provided, systems, method and non-transitory computer readable media for accessing a key value pair stored in a solid state drive (SSD) memory, the method may include calculating, by a SSD memory controller and based on an input key, a first bucket identifier and a first inter-bucket value; determining a block cluster that stores the key pair value, based on the first bucket identifier, the first inter-bucket value and first metadata of a first data structure; calculating, based on the input key, a second bucket identifier and a second inter-bucket value; determining at least one of a key pair value retrieval information and a representative key pair value retrieval information, based on the second bucket identifier, the second inter-bucket value and second metadata of a second data structure; wherein the second data structure is allocated to the block cluster; wherein the second metadata comprises second logical slots, a second slot locator and second collision separation metadata; wherein at le

Abstract: There may be provided, systems, method and non-transitory computer readable media for accessing a key value pair stored in a solid state drive (SSD) memory, the method may include calculating, by a SSD memory controller and based on an input key, a first bucket identifier and a first inter-bucket value; determining a block cluster that stores the key pair value, based on the first bucket identifier, the first inter-bucket value and first metadata of a first data structure; calculating, based on the input key, a second bucket identifier and a second inter-bucket value; determining key pair value retrieval information, based on the second bucket identifier, the second inter-bucket value and second metadata of a second data structure; wherein the second data structure is allocated to the block cluster; and retrieving at least the value of the key pair value based on the key pair value retrieval information.