Abstract: Methods and systems for managing storage of data in a distributed system are disclosed. To manage storage of data in a distributed system, a data processing system may include a network interface controller (NIC). The network interface controller may present emulated storages that may be used for data storage. The emulated storage devices may utilize storage resources of storage devices. The storage devices may be remote to the NIC. To reduce communication bandwidth and/or use of resources of the storage devices, the NIC and/or NICs of other data processing systems may implemented a distributed cache for data stored in the storage devices. The NICs may implement a method of managing the distributed cache to maintain synchronization between the distributed cache and the data stored in the storage devices.

Abstract: Methods and systems for managing storage of data in a distributed system are disclosed. To manage storage of data in a distributed system, a data processing system may include a network interface controller (NIC). The network interface controller may present emulated storages that may be used for data storage. The emulated storage devices may utilize storage resources of storage devices. The storage devices may be remote to the NIC. To reduce communication bandwidth and/or use of resources of the storage devices, the NIC and/or NICs of other data processing systems may implemented a distributed cache for data stored in the storage devices. The NICs may implement a method of managing the distributed cache to maintain synchronization between the distributed cache and the data stored in the storage devices.

Abstract: Methods and systems for managing storage of data in a distributed system is disclosed. To manage storage of data in a distributed system, a data processing system may include a network interface controller (NIC). The NIC may present emulated storages that may be used for data storage. The emulated storage devices may utilize storage resources of storage devices. The NIC and the storage devices may implement a distributed deduplication process. The NIC may segment data into chunks and obtain fingerprints of the chunks. The fingerprints may be provided to the storage which may check the fingerprints against fingerprints of already stored chunks. The storage may request the chunks corresponding to the fingerprints that did not match any fingerprints of the already stored chunks. The NIC may provide only those requested chunks to the storage before discarding all of the chunks.

Abstract: A host can include a programmable network interface card (NIC) or “Smart NIC” which accesses host-local drives hidden from a host processor. One configuration can include a switch with a one logical partition including the NIC as a root complex (RC) and the local drives as end points (EPs), and with another logical partition including the host processor as an RC and the NIC as an EP. A second configuration can include the NIC and switch directly connected to the host processor with an access control component (ACC) configured on switch ports connected to the local drives. A third configuration can include the NIC and local drives directly connected to the host processor with the ACC configured on host processor ports connected to the local drives. The NIC can use a multi-layer driver to communicate with the ACC and local drives hidden behind the ACC.

Abstract: Traditionally, servers are interconnected inside a data center using regular network cards. It is desired that high-available network-attached storage arrays have the feature of non-disruptive upgrade (NDU) for software and firmware, while one or more applications are still running IO. With the emergence of SmartNICs, there are many functions available now to SmartNIC that may enhance server and entire solution capabilities. Since SmartNIC is a new emerging technology, there are no adequate solutions currently for NDU while running I/O. The present patent document discloses embodiments for upgrading the SmartNIC software without disruption to the host applications. A shared namespace may be implemented inside an emulated NVMe/PCIe device, such as a data processing unit (DPU) or infrastructure processing unit (IPU), such that multiple instances may be enabled to run both old and new target emulation SPDK-based software together using multiple paths to achieve SmartNIC storage NDU.

Abstract: Methods and systems for managing storage of data in a distributed system are disclosed. To manage storage of data in a distributed system, a data processing system may include a network interface controller (NIC). The NIC may present emulated storages that may be used for data storage. The emulated storage devices may be implemented with a storage pipeline that uses any number of storage devices that may be local or remote to the data processing system. The computing resources of the data processing system may view the emulated storage as a local device. The NIC may use its storage pipeline to service its own storage needs. By doing so, the NIC may improve the likelihood that its hosted applications have sufficient storage service access.

Abstract: A method, computer program product, and computing system for deploying a storage processor of a storage system as a target of a non-volatile memory express (NVMe) over fabric (NVMe-oF) network. One or more NVMe storage devices coupled to the storage processor may be identified, thus defining one or more local NVMe storage devices. A smart network interface card may be coupled to the NVMe-oF network. The smart network interface card may be provided with access to the one or more local NVMe storage devices via the NVMe-oF network.

Abstract: Methods and systems for managing storage of data in a distributed system are disclosed. To manage storage of data in a distributed system, a data processing system may include a network interface controller (NIC). The NIC may present emulated storages that may be used for data storage. The emulated storage devices may be implemented with a storage pipeline that uses any number of storage devices that may be local or remote to the data processing system. The computing resources of the data processing system may view the emulated storage as a local device. The NIC may use its storage pipeline to service its own storage needs. By doing so, the NIC may improve the likelihood that its hosted applications have sufficient storage service access.

Abstract: Methods and systems for managing storage of data in a distributed system is disclosed. To manage storage of data in a distributed system, a data processing system may include a network interface controller (NIC). The NIC may present emulated storages that may be used for data storage. The emulated storage devices may utilize storage resources of storage devices. The NIC and the storage devices may implement a distributed deduplication process. The NIC may segment data into chunks and obtain fingerprints of the chunks. The fingerprints may be provided to the storage which may check the fingerprints against fingerprints of already stored chunks. The storage may request the chunks corresponding to the fingerprints that did not match any fingerprints of the already stored chunks. The NIC may provide only those requested chunks to the storage before discarding all of the chunks.

Abstract: Methods and systems for managing storage of data in a distributed system are disclosed. To manage storage of data in a distributed system, a data processing system may include a network interface controller (NIC). The network interface controller may present emulated storages that may be used for data storage. The emulated storage devices may utilize storage resources of storage devices. Overtime, the data stored in various data storages may be migrated between the data storages. To migrate the data, the NIC may manage the migration process. To do so, the NIC may obtain various access requests (e.g., writes, reads) and process them in a manner that allows for the migration to be transparent. By doing so, migrations that are transparent to compute resources of data processing systems may be completed.

Abstract: Traditionally, servers are interconnected inside a data center using regular network cards. It is desired that high-available network-attached storage arrays have the feature of non-disruptive upgrade (NDU) for software and firmware, while one or more applications are still running IO. With the emergence of SmartNICs, there are many functions available now to SmartNIC that may enhance server and entire solution capabilities. Since SmartNIC is a new emerging technology, there are no adequate solutions currently for NDU while running I/O. The present patent document discloses embodiments for upgrading the SmartNIC software without disruption to the host applications. A shared namespace may be implemented inside an emulated NVMe/PCIe device, such as a data processing unit (DPU) or infrastructure processing unit (IPU), such that multiple instances may be enabled to run both old and new target emulation SPDK-based software together using multiple paths to achieve SmartNIC storage NDU.

Abstract: A host can include a programmable network interface card (NIC) or “Smart NIC” which accesses host-local drives hidden from a host processor. One configuration can include a switch with a one logical partition including the NIC as a root complex (RC) and the local drives as end points (EPs), and with another logical partition including the host processor as an RC and the NIC as an EP. A second configuration can include the NIC and switch directly connected to the host processor with an access control component (ACC) configured on switch ports connected to the local drives. A third configuration can include the NIC and local drives directly connected to the host processor with the ACC configured on host processor ports connected to the local drives. The NIC can use a multi-layer driver to communicate with the ACC and local drives hidden behind the ACC.

Abstract: A method, computer program product, and computing system for deploying a storage processor of a storage system as a target of a non-volatile memory express (NVMe) over fabric (NVMe-oF) network. One or more NVMe storage devices coupled to the storage processor may be identified, thus defining one or more local NVMe storage devices. A smart network interface card may be coupled to the NVMe-oF network. The smart network interface card may be provided with access to the one or more local NVMe storage devices via the NVMe-oF network.

Abstract: A method of managing a storage appliance is provided. The method includes (a) partitioning a cache of the storage appliance at least into multiple regions dedicated to respective storage drives of a plurality of storage drives of the storage appliance; (b) in response to the storage appliance receiving a first storage command directed to a first storage drive, allocating space for fulfillment of the first storage command within the region of cache dedicated to the first storage drive; (c) in response to the storage appliance receiving a second storage command directed to a second storage drive, allocating space for fulfillment of the second storage command within the region of cache dedicated to the second storage drive; and (d) fulfilling, by the storage appliance, the first and second storage commands by moving data to and from their respective allocated space in the cache. An apparatus, system, and computer program product for performing a similar method are also provided.

Abstract: A method, computer program product, and computing system for generating one or more unique network addresses for one or more storage array enclosures of a storage system. The one or more unique network addresses may be advertised to one or more storage processors of the storage system. The one or more storage processors may be configured to communicate with a plurality of storage devices of the one or more storage array enclosures based upon, at least in part, the one or more unique network addresses of the one or more storage array enclosures.

Abstract: A method, computer program product, and computing system for receiving, at a first baseboard management controller (BMC) of a first node, a fence request from a central processing unit (CPU) of the first node. The first BMC of the first node may communicate with a second BMC of a second node, wherein the first node and the second node are communicatively coupled to a plurality of storage drives. The first BMC and the second BMC may determine which node of the first node and the second node to fence from the plurality of storage drives. One of the first node and the second node may be fenced, via one of the first BMC and the second BMC, from the plurality of storage drives, thus defining a fenced node.

Abstract: A method of managing a storage appliance is provided. The method includes (a) partitioning a cache of the storage appliance at least into multiple regions dedicated to respective storage drives of a plurality of storage drives of the storage appliance; (b) in response to the storage appliance receiving a first storage command directed to a first storage drive, allocating space for fulfillment of the first storage command within the region of cache dedicated to the first storage drive; (c) in response to the storage appliance receiving a second storage command directed to a second storage drive, allocating space for fulfillment of the second storage command within the region of cache dedicated to the second storage drive; and (d) fulfilling, by the storage appliance, the first and second storage commands by moving data to and from their respective allocated space in the cache. An apparatus, system, and computer program product for performing a similar method are also provided.

Abstract: A storage system comprises a disk array enclosure comprising an enclosure controller, a cache comprising a metadata journal, a plurality of data storage devices and a plurality of metadata storage devices. The enclosure controller is configured to write a stripe metadata page to the metadata storage devices that corresponds to a stripe of data stored on the data storage devices and to determine that the write of the stripe metadata page failed for a first metadata storage device. The enclosure controller is configured to add an entry to the metadata journal based on the determination that the write failed. The entry comprises an indication of the first metadata storage device and the stripe of data. The enclosure controller is configured to set an indication in a data structure associated with the disk array enclosure that the stripe metadata page has not been written to the first metadata storage device.

Abstract: A storage system comprises a plurality of enclosures and a storage controller. Each enclosure comprises at least one processing device and a plurality of drives configured in accordance with a redundant array of independent disks (RAID) arrangement. The storage controller obtains data pages associated with an input-output request, provides the data pages to a processing device of a given enclosure, and issues a command to the processing device to perform at least one operation based at least in part on the data pages. The processing device of the given enclosure receives the data pages from the storage controller, generates compressed data pages based at least in part on the received data pages, stores one or more of the compressed data pages on the plurality of drives according to the RAID arrangement and returns information associated with the storage of the compressed data pages to the storage controller.

Abstract: A method, computer program product, and computing system for generating one or more unique network addresses for one or more storage array enclosures of a storage system. The one or more unique network addresses may be advertised to one or more storage processors of the storage system. The one or more storage processors may be configured to communicate with a plurality of storage devices of the one or more storage array enclosures based upon, at least in part, the one or more unique network addresses of the one or more storage array enclosures.