Abstract: A system and method improve the performance of non-volatile memory storage by rebuilding, on the fly, “lost data” in response to a read request, which identifies data to be read or recovered, by identifying a parity data storage device in a set of data storage devices that contains parity corresponding to the identified data; sending a reconstruction request to a respective data storage device, which may be the parity data storage device or other data storage device in the system, to reconstruct the identified data, and receiving the identified data from the respective data storage device. The reconstruction request commands the respective data storage device to retrieve, via peer-to-peer read requests, from other data storage devices, data from one or more data blocks, and to reconstruct the identified data based on the retrieved data and parity data locally stored at the parity data storage device.

Abstract: A system and method improve the performance of non-volatile memory storage by offloading parity computations to facilitate high speed data transfers, including direct memory access (DMA) transfers, between a remote host and a non-volatile memory based storage system, such as a flash memory based data storage device (e.g., SSD). In conjunction with writing to non-volatile memory storage, a stripe map is used to target a selected data storage device for parity generation. All data of a stripe is transmitted to the selected data storage device to generate the parity and the generated parity is propagated from the selected data storage device to other data storage devices in the stripe. The data for the stripe may also be propagated from the selected data storage device to the other data storage devices in the stripe.

Abstract: A method of managing a storage system having one or more storage devices includes receiving a request to access, in a tiered data structure, a data object and one or more attributes of the data object and processing the access request. The request includes a first key value for the data object and respective second key values for the one or more attributes of the data object. Modified key values for the attributes of the data object are generated in accordance with at least a portion of the first key value and used to access tree entries for the data object attributes in one or more leaf nodes of the tiered data structure. A tree entry for the data object is also accessed from the same set of one or more leaf nodes.

Abstract: A host compiles code to perform a set of one or more database operations on target and embeds an indication of whether the target data is randomly accessed data or sequentially accessed data. The compiled code is transmitted to the compute engine inside a memory system that maintains a first portion of memory for storing sequentially accessed data and a second portion of memory for storing randomly accessed data. The memory system (e.g. SSD) maintains reduced size L2P tables in volatile working memory by maintaining coarse L2P tables in the working memory for use with sequentially accessed data and maintaining fine L2P tables in the working memory for use with randomly accessed data. The compute engine uses the compiled code to perform the set of one or more database operations on the target data using the working memory.

Abstract: A memory controller receives request(s) to perform a plurality of memory operations on a datastore. The memory controller batches, into a set of leaf node operations, memory operations of the plurality of memory operations that are associated with keys in a range of keys, where a leaf node in a tiered data structure stored in the datastore is assigned the range of keys. The memory controller determines whether a condition is satisfied, and, in accordance with a determination that the condition is satisfied: locates the leaf node in the datastore; stores a copy of the leaf node in a cache; modifies the cached copy of the leaf node according to the set of leaf node operations so as to obtain one or more modified leaf nodes; and writes the one or more modified leaf nodes to one or more locations in the datastore with one or more write operations.

Abstract: Systems, methods and/or devices are used to coalesce metadata and data writes via write serialization with device-level address remapping. In one aspect, a method of managing a storage system having one or more storage devices includes a serialized write operation to the storage system, in which a serialization segment accumulates data objects and mapping information until the segment is full, at which time the serialization segment is written to the storage system in a single contiguous write. As a result, the number of I/O operations is decreased from a minimum of two (one to write data and one to write updated mapping information) to a single write operation. Further, if the serialization segment contains existing valid data prior to accumulating data objects and mapping information, the valid data is moved to the beginning of the serialization segment using either a remap or xcopy operation.

Abstract: A method of managing a storage system having one or more storage devices includes a host-based garbage collection operation that includes identifying a logical stripe in accordance with data storage information stored at the host system, and enabling a process of coalescing valid data in a target logical stripe, the coalescing process including moving the valid data in the logical stripe and repacking valid logical addresses to the beginning of the target logical stripe, comprising the identified logical stripe or another logical stripe. Further, the use of an internal copy operation allows the host-based garbage collection operation to occur without transferring data back to the host, thus minimizing the number of I/O operations between the host and storage devices. Additionally, use of the host-based garbage collection operation allows device-level garbage collection to be minimized, and more sophisticated garbage collection algorithms (e.g., matching the current workload) to be used.

Abstract: Systems, methods and/or devices are used for efficient implementation of optimized host-based garbage collection strategies using xcopy and arrays of flash devices. In one aspect, a method of managing a storage system having one or more storage devices includes a host-based garbage collection operation that includes identifying two or more logical stripes in accordance with data storage information stored at the host system, and enabling a process of coalescing valid data in the two or more logical stripes. Further, the use of an internal copy operation (e.g., xcopy), allows the host-based garbage collection operation to occur without transferring data back to the host, thus minimizing the number of I/O operations between the host and storage devices. Additionally, use of the host-based garbage collection operation allows more sophisticated garbage collection algorithms (e.g., matching the current workload) to be used, and ensures that multiple logical stripes are available to write data.

Abstract: Systems, methods and/or devices are used to perform memory-efficient mapping of block/object addresses. In one aspect, a method of managing a storage system having one or more storage devices includes a tiered data structure in which each node has a logical ID and entries in the nodes reference other nodes in the tiered data structure using the logical IDs. As a result, when a child node is updated and stored to a new location, but retains its logical ID, its parent node does not need to be updated, because the logical ID in the entry referencing the child node remains unchanged. Further, the storage system uses a secondary mapping table to translate the logical IDs to the corresponding physical locations of the corresponding nodes. Additionally, the secondary mapping table is cached in volatile memory, and as a result, the physical location of a required node is determined without accessing non-volatile memory.

Abstract: Systems, methods, and/or devices are used to store metadata in a storage system. In one aspect, a first user space module sends a logical memory request to a memory management module of a kernel space module. The logical memory request includes data and metadata. A second user space module obtains the metadata of the logical memory request. A storage engine of the second user space module determines, in accordance with the obtained metadata, a location in non-volatile memory for the data. A second user space module generates a physical memory request including an indication of the non-volatile memory for the data. The second user space module transmits the physical memory request to the kernel space memory management module.

Abstract: A computer system detects a request, from a requestor, to access a first data object stored in a tiered data structure that includes internal nodes and leaf nodes. In response to detecting the request to access the first data object, the computer system retrieves a leaf node that includes the first data object and locks the leaf node that includes the first data object. While the leaf node that includes the first data object is locked, the computer system transmits, to the requestor, a first conditional-update communication and detecting a first conditional-update response in response to the first conditional-update communication. In response to detecting the first conditional-update response corresponding to the first data object, the computer system performs one or more operations based on the first conditional-update response corresponding to the first data object before unlocking the leaf node that includes the first data object.

Abstract: A computer system detects a request to access a first data object stored in a tiered data structure, that includes internal nodes and leaf nodes, where data objects in the leaf nodes include unique key information and corresponding values, and the first data object is uniquely identified by a first key. In response to detecting the request to access the first data object, the computer system retrieves a leaf node that includes the first data object and identifies the first data object in the leaf node by combining unique key information of the first data object with a key prefix that is stored separately in the leaf node to generate a combined key and determining that the combined key matches the first key that uniquely identifies the first data object. After identifying the first data object, the computer system provides access to the first data object.

Abstract: A method of managing a storage system having one or more storage devices includes receiving a request to access, in a tiered data structure, a data object and one or more attributes of the data object and processing the access request. The request includes a first key value for the data object and respective second key values for the one or more attributes of the data object. Modified key values for the attributes of the data object are generated in accordance with at least a portion of the first key value and used to access tree entries for the data object attributes in one or more leaf nodes of the tiered data structure. A tree entry for the data object is also accessed from the same set of one or more leaf nodes.

Abstract: Embodiments of a system that includes an array of chip modules (CMs) is described. In this system, a given CM in the array includes a semiconductor die that is configured to communicate data signals with one or more adjacent CMs through electromagnetic proximity communication using proximity connectors. Note that the proximity connectors are proximate to a surface of the semiconductor die. Moreover, the given CM is configured to communicate optical signals with other CMs through an optical signal path using optical communication, and the optical signals are encoded using wavelength-division multiplexing (WDM).

Abstract: A computer system detects a request to access a first data object stored in a tiered data structure that includes internal nodes and leaf nodes. In response to detecting the request, the computer system determines whether the first data object is stored in a cache that includes a plurality of data objects from the tiered data structure that are stored separately from their corresponding leaf node in the cache. When the first data object is stored in the cache, the computer system returns the first data object from the cache and when the first data object is not stored in the cache, the computer system traverses the tiered data structure to a leaf node that includes the first data object and returns the first data object from the leaf node for the first data object in the tiered data structure.

Abstract: A computer system detects a request to access a first data object stored in a tiered data structure that includes internal nodes and leaf nodes. In response to detecting the request, the computer system determines whether the first data object is stored in a cache that includes a plurality of data objects from the tiered data structure that are stored separately from their corresponding leaf node in the cache. When the first data object is stored in the cache, the computer system returns the first data object from the cache and when the first data object is not stored in the cache, the computer system traverses the tiered data structure to a leaf node that includes the first data object and returns the first data object from the leaf node for the first data object in the tiered data structure.

Abstract: A memory controller receives request(s) to perform a plurality of memory operations on a datastore. The memory controller batches, into a set of leaf node operations, memory operations of the plurality of memory operations that are associated with keys in a range of keys, where a leaf node in a tiered data structure stored in the datastore is assigned the range of keys. The memory controller determines whether a condition is satisfied, and, in accordance with a determination that the condition is satisfied: locates the leaf node in the datastore; stores a copy of the leaf node in a cache; modifies the cached copy of the leaf node according to the set of leaf node operations so as to obtain one or more modified leaf nodes; and writes the one or more modified leaf nodes to one or more locations in the datastore with one or more write operations.

Abstract: In response to receiving a request to perform a transaction with two or more memory operations on one or more tiered data structures, the memory controller: writes a start transaction record to the log stream including a transaction identifier corresponding to the transaction; and performs the two or more memory operations. For a first memory operation associated with a key, the memory controller: writes a new data object in a datastore; assigns, in a key-map, a location of the new data object to the key; maintains an old data object in the datastore whose location was previously assigned to the key; and writes an operation commit record to a log stream upon completion of the first memory operation. In accordance with a determination that the two or more memory operations are complete, the memory controller writes a transaction commit record to the log stream including the transaction identifier.

Abstract: A computer system detects a request, from a requestor, to access a first data object stored in a tiered data structure that includes internal nodes and leaf nodes. In response to detecting the request to access the first data object, the computer system retrieves a leaf node that includes the first data object and locks the leaf node that includes the first data object. While the leaf node that includes the first data object is locked, the computer system transmits, to the requestor, a first conditional-update communication and detecting a first conditional-update response in response to the first conditional-update communication. In response to detecting the first conditional-update response corresponding to the first data object, the computer system performs one or more operations based on the first conditional-update response corresponding to the first data object before unlocking the leaf node that includes the first data object.

Abstract: A computer system detects a request to access a first data object stored in a tiered data structure, that includes internal nodes and leaf nodes, where data objects in the leaf nodes include unique key information and corresponding values, and the first data object is uniquely identified by a first key. In response to detecting the request to access the first data object, the computer system retrieves a leaf node that includes the first data object and identifies the first data object in the leaf node by combining unique key information of the first data object with a key prefix that is stored separately in the leaf node to generate a combined key and determining that the combined key matches the first key that uniquely identifies the first data object. After identifying the first data object, the computer system provides access to the first data object.