Abstract: A distributed data storage system can be configured with a host connected to a first device and an initialization module that performs a default initialization procedure on the first device in response to detecting the first device has an unknown manufacturing origin. Conducting the default initialization procedure may allow the first device to service data access requests from the host. In response to the connection of a second device to the host and initialization module, a manufacturing origin of the second device is identified with the initialization module before the initialization module customizes the default initialization procedure to a custom procedure that is executed to allow the second device to satisfy a background operation prescribed by the initialization module.

Abstract: A data storage system may connect a host to a device and a hardware module that are utilized to satisfy at least one host-generated data access request to the device. A portion of the device is set as dormant by the hardware module prior to identifying an origin of the device with the hardware module during the satisfaction of the at least one host-generated data access request. In response to the identified origin, the previously dormant portion of the device is activated and subsequently utilized to execute a task assigned by the hardware module.

Abstract: A data storage system may connect a host to a device and a hardware module that are utilized to satisfy at least one host-generated data access request to the device. A portion of the device is set as dormant by the hardware module prior to identifying an origin of the device with the hardware module during the satisfaction of the at least one host-generated data access request. In response to the identified origin, the previously dormant portion of the device is activated and subsequently utilized to execute a task assigned by the hardware module.

Abstract: A distributed data storage system may connect a host to a device and a dedication module. The dedication module can identify a manufacturing origin of the device before dedicating a type of host-generated data access request to the device. The type of host-generated data access request is chosen by the dedication module in response to an operating capability correlated to the device by the dedication module based on the identified manufacturing origin. At least one host-generated request of the dedicated type is then completed with the device.

Abstract: A distributed data storage system can be configured with a host connected to a device and a distribution module. The distribution module identifies a manufacturing origin of the device and diverts a system operation from an upstream component connected to the distribution module to the device in response to the identified manufacturing origin of the device. The manufacturer installed operating parameters of the device are then used to complete the system operation.

Abstract: Apparatus and method for executing hidden computational functions in a distributed data processing environment. In some embodiments, a trust boundary includes a target device such as a storage device, and a source device such as a client device in a computer network. A storage device processor executes a hidden command function (HCF) routine to accumulate HCF output data in a local cache responsive to an HCF command received from the client device over a data interface. The processor further establishes a smaller retention boundary within the trust boundary that includes the storage device and excludes the client device. The HCF output data are stored locally in a non-volatile memory (NVM) of the storage device while not transferring any portion of the HCF output data outside the retention boundary, including to the client device. The HCF routine can update a block-chain ledger or take some other form to provide data security.

Abstract: Method and apparatus for offloading upstream processing tasks to peer groups of downstream data storage devices. A peer control circuit forms a peer group of storage devices in response to a detected processing bottleneck associated with a network controller. One of the storage devices in the peer group is designated as a primary device, and is responsible for interface communications, for subdividing the processing task for execution by secondary devices in the peer group, and coordinating overall execution. The peer group and the processing task are selected to avoid or minimize the processing bottleneck at the network controller level while maintaining ongoing data transfer performance at the storage device level. A list of available device resources and capabilities may be maintained by the peer control circuit. Offloaded tasks can include data rebuilds, cryptographic functions, new device authentication operations, and the like. Multiple overlapping peer groups can be formed as needed.

Abstract: A distributed data storage system can be configured with a host connected to a first device and an initialization module that performs a default initialization procedure on the first device in response to detecting the first device has an unknown manufacturing origin. Conducting the default initialization procedure may allow the first device to service data access requests from the host. In response to the connection of a second device to the host and initialization module, a manufacturing origin of the second device is identified with the initialization module before the initialization module customizes the default initialization procedure to a custom procedure that is executed to allow the second device to satisfy a background operation prescribed by the initialization module.

Abstract: Apparatus and method for maintaining real-time coherency between a local cache of a target device and a client cache of a source device during execution of a distributed computational function. In some embodiments, a source device, such as a host computer, is coupled via a network interface to a target device, such as a data storage device. A storage compute function (SCF) command is transferred from the source device to the target device. A local cache of the target device accumulates output data during the execution of an associated SCF over an execution time interval. Real-time coherency is maintained between the contents of the local cache and a client cache of the source device, so that the client cache retains continuously updated copies of the contents of the local cache during execution of the SCF. The coherency can be carried out on a time-based granularity or an operational granularity.

Abstract: The technology disclosed herein pertains to a system and method for scaling storage using peer-to-peer NVMe communication, the method including selecting one of a plurality of NVMe devices as principal device to control communication with a host via a PCI switch, designating remainder of the plurality of NVMe devices as subordinate devices, and controlling the communication between the host and the subordinate devices using a PCI P2P DMA between the principal device and the subordinate devices.

Abstract: Methods, apparatuses, and computer-readable media for streaming arbitrarily large amounts of data through computational storage programs of a computational storage device. A computational storage device comprises a storage media, a computational storage processor, and a controller. A firmware of the controller comprises a plurality of streaming drivers, each associated with a data source or data destination of the storage device. The firmware further comprises a buffer abstraction layer operable to read data from a data source through an associated ingress streaming driver of the plurality of streaming drivers to provide a source data stream for a computational storage program executing on the computational storage processor. The buffer abstraction layer is further operable to receive a destination data stream from the computational storage program and write data to a data destination through an associated egress streaming driver of the plurality of streaming drivers.

Abstract: A system includes a storage device and a computational storage processor. The storage device includes media. The computational storage processor is configured to, after issuance of a single command from a host device, receive data corresponding to the command, process the data as the data is received using a filter program and provide results data from the processed data.

Abstract: A system includes a storage device and a computational storage processor. The storage device includes media. The computational storage processor is configured to, after issuance of a single command from a host device, receive data corresponding to the command, process the data as the data is received using a filter program and provide results data from the processed data.

Abstract: A multi-function NVMe subsystem includes a plurality of primary controllers, and a plurality of queue resources. The multi-function NVMe subsystem also includes a plurality of policies with each different policy of the plurality of policies differently dictating how the plurality of queue resources is divided amongst different primary controllers of the plurality of primary controllers.

Abstract: Methods, apparatuses, and computer-readable media for streaming arbitrarily large amounts of data through computational storage programs of a computational storage device. A computational storage device comprises a storage media, a computational storage processor, and a controller. A firmware of the controller comprises a plurality of streaming drivers, each associated with a data source or data destination of the storage device. The firmware further comprises a buffer abstraction layer operable to read data from a data source through an associated ingress streaming driver of the plurality of streaming drivers to provide a source data stream for a computational storage program executing on the computational storage processor. The buffer abstraction layer is further operable to receive a destination data stream from the computational storage program and write data to a data destination through an associated egress streaming driver of the plurality of streaming drivers.

Abstract: Embodiments of the present invention provide for creating and using persistent connections in SAS networks. A persistent connection may be a connection that persists for longer than the usual SAS connection. More specifically, it is a connection that is not subject to periodic tear downs by SAS devices according to existing SAS protocols (such as, by using CLOSE or BREAK primitives). Instead, persistent connections may be removable bra link reset Persistent connections may be used in situations in which the overhead associated with the usual tear down and re-establishment of connections in a SAS network may be considered too high in comparison with its intended benefits. Persistent connections way also be used to provide virtual direct attachment between two different SAS connected devices or between a SAS connected device and an expander.

Abstract: A network device comprising a first attach point, a second attach point, a switch and persistent connection logic is provided. The first attach point may connect the network device to a first link, and the second attach point may connect the network device to a second link. The switch may connect the first attach point to the second attach point. The persistent connection logic may create a persistent connection between a first network element and a second network element, where the persistent connection comprises the network device, the first link and the second link. The network device may also implement a non-persistent connection between two network elements, where the non-persistent connection may comprises the network device.

Abstract: A network device comprising a first attach point, a second attach point, a switch and persistent connection logic is provided. The first attach point may connect the network device to a first link, and the second attach point may connect the network device to a second link. The switch may connect the first attach point to the second attach point. The persistent connection logic may create a persistent connection between a first network element and a second network element, where the persistent connection comprises the network device, the first link and the second link. The network device may also implement a non-persistent connection between two network elements, where the non-persistent connection may comprises the network device.

Abstract: Embodiments of the present invention provide for creating and using persistent connections in SAS networks. A persistent connection may be a connection that persists for longer than the usual SAS connection. More specifically, it is a connection that is not subject to periodic tear downs by SAS devices according to existing SAS protocols (such as, by using CLOSE or BREAK primitives). Instead, persistent connections may be removable by a link reset. Persistent connections may be used in situations in which the overhead associated with the usual tear down and re-establishment of connections in a SAS network may be considered too high in comparison with its intended benefits. Persistent connections may also be used to provide virtual direct attachment between two different SAS connected devices or between a SAS connected device and an expander.

Abstract: Described herein is an improved mechanism for bridging between SAS and SATA drives based upon existing SAS expanders in a SAS domain. In particular, a bridge capable of translating between SAS and SATA protocols is embedded in or coupled to an expander. When a SAS initiator request is received at the expander, the expander can route the request, based on a routing table, either directly to a destination SAS device or to the bridge for necessary translation before it is transmitted to a destination SATA drive. The routing table includes corresponding relationships between all SAS addresses and Phys through which those SAS and SATA devices are attached to the expander. SATA devices can be virtualized in the expander through a few assigned addresses in the routing table in a SAS discovery process.