Abstract: A method for use in a computing device, the method comprising: detecting a request to transmit data, the data including a plurality of data portions, each of the data portions being associated with a respective remote device that the data portion is destined for; identifying a list of addresses that is associated with the request, each of the addresses in the list corresponding to the respective remote device of a different one of the plurality of data portions; detecting whether the request is local; when the request is not local, transmitting the data over a communications network; when the request is local transmitting each of the data portions in the plurality over the respective DMA link that couples the computing device with the data portion's respective remote device.

Abstract: A pair of compute nodes, each having a separate PCIe root complex, are interconnected by a PCIe Non-Transparent Bridge (NTB). An instance of a NTB monitoring process is started for each root complex, and the CPU affinity of the NTB monitoring processes are set to cause each NTB monitoring process to be executed on CPU resources of each respective CPU root complex. The NTB monitoring process on a given root complex is allowed to sleep until a triggering event occurs that causes the NTB monitoring process to wake and determine the state of the NTB. One such triggering event is a failure of an atomicity algorithm on the compute node to obtain a lock on peer memory in connection with implementing an atomic read operation on the peer memory over the NTB.

Abstract: Architectures and techniques are described that can provide security or threat detection for a data storage system. Threat detection can be identified and potentially blocked prior to requested customer workloads reaching a backend storage device and can thus effectively be achieved in real-time. Techniques utilized herein can leverage an offload capability that operates to offload certain processing from a central processing unit (CPU) to a data processing unit (DPU). A long short-term memory (LSTM) model can be executed in the DPU to detect potential threats in real-time without consuming CPU resources.

Abstract: A deterministic hardware indictment process is used to indict one Compute Node (CN) of a pair of CNs in response to occurrence of a fatal error on a Peripheral Component Interconnect Express (PCIe) Non-Transparent Bridge (NTB) interconnecting the two CN. Status information is exchanged between the CNs on a communication medium that is separate from the NTB. The indictment process is run locally on each CN based on whatever information is available to the CN. When the CN does not receive status information from the other CN, the CN runs a self-indictment check. When status information is received, and only one CN reports an error, the indictment process indicts the compute node that reported the error. If both CNs report errors, an error severity comparison process is used to select a CN to be indicted. If the reported errors are equally severe, a default CN is indicted.

Abstract: A method for use in a computing device, the method comprising: detecting a request to transmit data, the data including a plurality of data portions, each of the data portions being associated with a respective remote device that the data portion is destined for; identifying a list of addresses that is associated with the request, each of the addresses in the list corresponding to the respective remote device of a different one of the plurality of data portions; detecting whether the request is local; when the request is not local, transmitting the data over a communications network; when the request is local transmitting each of the data portions in the plurality over the respective DMA link that couples the computing device with the data portion's respective remote device.

Abstract: A storage engine has a pair of compute nodes, each compute node having a separate PCIe root complex and attached memory. The PCIe root complexes are interconnected by multiple Non-Transparent Bridge (NTB) links. The NTB resources are unequally shared, such that host IO devices are required to use a first subset of the NTB links to implement memory access operations on the memory of the peer compute node, whereas storage software memory access operations are able to be implemented on all of the NTB links. A NTB link arbitration system arbitrates usage of the first and second subsets of NTB links by the storage software, to distribute subsets of the storage software memory access operations on peer memory to the first and second subsets of NTB links, while causing all host IO device memory access operations on peer memory to be implemented on the first set of NTB links.

Abstract: A pair of compute nodes, each having a separate PCIe root complex, are interconnected by a PCIe Non-Transparent Bridge (NTB). An instance of a NTB monitoring process is started for each root complex, and the CPU affinity of the NTB monitoring processes are set to cause each NTB monitoring process to be executed on CPU resources of each respective CPU root complex. The NTB monitoring process on a given root complex is allowed to sleep until a triggering event occurs that causes the NTB monitoring process to wake and determine the state of the NTB. One such triggering event is a failure of an atomicity algorithm on the compute node to obtain a lock on peer memory in connection with implementing an atomic read operation on the peer memory over the NTB.

Abstract: A deterministic hardware indictment process is used to indict one Compute Node (CN) of a pair of CNs in response to occurrence of a fatal error on a Peripheral Component Interconnect Express (PCIe) Non-Transparent Bridge (NTB) interconnecting the two CN. Status information is exchanged between the CNs on a communication medium that is separate from the NTB. The indictment process is run locally on each CN based on whatever information is available to the CN. When the CN does not receive status information from the other CN, the CN runs a self-indictment check. When status information is received, and only one CN reports an error, the indictment process indicts the compute node that reported the error. If both CNs report errors, an error severity comparison process is used to select a CN to be indicted. If the reported errors are equally severe, a default CN is indicted.

Abstract: A storage engine has a pair of compute nodes, each compute node having a separate PCIe root complex and attached memory. The PCIe root complexes are interconnected by multiple Non-Transparent Bridge (NTB) links. The NTB resources are unequally shared, such that host IO devices are required to use a first subset of the NTB links to implement memory access operations on the memory of the peer compute node, whereas storage software memory access operations are able to be implemented on all of the NTB links. A NTB link arbitration system arbitrates usage of the first and second subsets of NTB links by the storage software, to distribute subsets of the storage software memory access operations on peer memory to the first and second subsets of NTB links, while causing all host IO device memory access operations on peer memory to be implemented on the first set of NTB links.

Abstract: Disclosed are techniques for compressing data in a data storage system comprising searching a cluster of nearest neighbors, wherein the cluster has been created using a locality sensitive hashing algorithm, to determine if a data block can be compressed. In alternate embodiments, nearest neighbor clusters can be formed using unsupervised learning. Additionally, nearest neighbors can also be formed in alternate embodiments using one or more of the following algorithms: a k-means clustering algorithm, a k-medoids clustering algorithm, a mean shift algorithm, a generalized method of moment (GMM) algorithm, or a density based spatial clustering of applications with noise (DBSCAN) algorithm.

Abstract: One or more aspects of the present disclosure relate to controlling threads across several processors. For example, at least one input/output (IO) workflow message can be stored in a storage array's hardware queue. Additionally, an IO workflow message can be read from the hardware queue. Further, a local thread wake-up or an interrupt-wakeup operation can be performed based on a target of the IO workflow message.

Abstract: A host IO devices directly implements host read operations on both local memory, and on peer memory via a PCIe non-transparent bridge. When a host read operation is received by a host IO device from a host, the host IO device uses an API to obtain the physical address of the requested data on the peer memory, and generates a PCIe Transaction Layer Packet (TLP) addressed to the address in the peer memory. The TLP addressed to an address in the peer memory is passed over the NTB to the peer compute node to retrieve the data stored in the addressed slot of peer memory. The requested data is returned to the host IO device over the NTB, stored in a buffer, and read out to the host to directy respond to the host read operation.

Abstract: Techniques for data deduplication may include: receiving write operations that write first data; partitioning the first data into a plurality of data portions; generating, using a first hash function, a plurality of data deduplication hash values for the plurality of data portions, wherein a first data deduplication hash value of the plurality of data deduplication hash values is produced by said generating for a first of the plurality of data portions; performing first processing using a Bloom filter to determine whether the first data deduplication hash value has a corresponding first entry in a data store of deduplication hash values; and responsive to the first processing determining the first data deduplication hash value does not have the corresponding first entry in the data store of deduplication hash values, performing second processing, said second processing including adding the corresponding first entry in the data store of deduplication hash values.

Abstract: Disclosed are techniques for data deduplication, which include methods, systems, or computer products for reducing data redundancy in a data storage system comprising searching a cluster of nearest neighbors, wherein the cluster has been created using a locality sensitive hashing algorithm, to determine if a data block has been stored in the data storage system prior to writing the data block. In alternate embodiments, the nearest neighbor clusters could be created using one or more of the following algorithms: k-means clustering algorithm, a k-medoids clustering algorithm, a mean shift algorithm, a generalized method of moment (GMM) algorithm, or a density based spatial clustering of applications with noise (DBSCAN) algorithm.

Abstract: First and second virtual storage controllers represent first and second physical storage controllers that provide access to different physical storage devices that back first and second virtual disks. One or more computing nodes host a virtual storage array that includes paired virtual storage directors. Each virtual storage director has access to both the first and second virtual storage controllers. Thus, the virtual disks may be presented to the storage directors as dual-ported disks even if the physical storage devices are single-ported. The virtual storage directors and virtual storage controllers may each be hosted by a separate virtual machine or instantiated in a separate container.

Abstract: A storage array is implemented as a virtual machine on a server. The virtual storage array includes a virtual engine with virtual directors and virtualized processor and memory resources. Embedded storage software converts Input/Output requests into Input/Output requests that can be serviced by the virtual machine. A virtual backend driver enables the virtual storage directors to access a set of virtual drives. A virtualized shared memory driver provides emulated input/output, XOR, direct memory access and copy services for send and receive threads.

Abstract: Disclosed are techniques for compressing data in a data storage system comprising searching a cluster of nearest neighbors, wherein the cluster has been created using a locality sensitive hashing algorithm, to determine if a data block can be compressed. In alternate embodiments, nearest neighbor clusters can be formed using unsupervised learning. Additionally, nearest neighbors can also be formed in alternate embodiments using one or more of the following algorithms: a k-means clustering algorithm, a k-medoids clustering algorithm, a mean shift algorithm, a generalized method of moment (GMM) algorithm, or a density based spatial clustering of applications with noise (DBSCAN) algorithm.

Abstract: Disclosed are techniques for data deduplication, which include methods, systems, or computer products for reducing data redundancy in a data storage system comprising searching a cluster of nearest neighbors, wherein the cluster has been created using a locality sensitive hashing algorithm, to determine if a data block has been stored in the data storage system prior to writing the data block. In alternate embodiments, the nearest neighbor clusters could be created using one or more of the following algorithms: k-means clustering algorithm, a k-medoids clustering algorithm, a mean shift algorithm, a generalized method of moment (GMM) algorithm, or a density based spatial clustering of applications with noise (DBSCAN) algorithm.

Abstract: Techniques for data deduplication may include: receiving write operations that write first data; partitioning the first data into a plurality of data portions; generating, using a first hash function, a plurality of data deduplication hash values for the plurality of data portions, wherein a first data deduplication hash value of the plurality of data deduplication hash values is produced by said generating for a first of the plurality of data portions; performing first processing using a Bloom filter to determine whether the first data deduplication hash value has a corresponding first entry in a data store of deduplication hash values; and responsive to the first processing determining the first data deduplication hash value does not have the corresponding first entry in the data store of deduplication hash values, performing second processing, said second processing including adding the corresponding first entry in the data store of deduplication hash values.

Abstract: Techniques for data deduplication may include: receiving write operations that write first data; partitioning the first data into a plurality of data portions; generating, using a first hash function, a plurality of data deduplication hash values for the plurality of data portions, wherein a first data deduplication hash value of the plurality of data deduplication hash values is produced by said generating for a first of the plurality of data portions; performing first processing using a Bloom filter to determine whether the first data deduplication hash value has a corresponding first entry in a data store of deduplication hash values; and responsive to the first processing determining the first data deduplication hash value does not have the corresponding first entry in the data store of deduplication hash values, performing second processing, said second processing including adding the corresponding first entry in the data store of deduplication hash values.