Abstract: Tiers of compression algorithms may be determined using compression information collected regarding compression ratios achieved for data sets using compression algorithms. Each tier may meet specified criteria regarding expected compression ratios achieved for a specified portion or number of data sets. Compression algorithms of each tier may be implemented by a different hardware device that may include hardware accelerators for the algorithms of the tier. Different tiers, and thus different hardware devices, achieve different levels of compression. A recommendation may be provided using compression information collected, such as from one of the hosts, regarding which hardware device to use for compression. The recommendation may be to purchase a license to use or whether to purchase a particular hardware device for compression. Compression information may be collected by a host that issues tagged I/Os providing a hint regarding what compression algorithm to use for the particular I/O operation data.

Abstract: A storage system may assign a different encryption key to each logical storage unit (LSU) of a storage system. For each LSU, the encryption key of the LSU may be shared only with host systems authorized to access data of the LSU. In response to a read request for a data portion received from a host application executing on the host system, encryption metadata for the data portion may be accessed. If it is determined from the encryption metadata that the data portion is encrypted, the data encryption metadata for the data portion may be further analyzed to determine the encryption key for the data portion. The data may be retrieved from the storage system, for example, by performance of a direct read operation. The retrieved data may be decrypted, and the decrypted data may be returned to the requesting application.

Abstract: Tiers of compression algorithms may be determined using compression information collected regarding compression ratios achieved for data sets using compression algorithms. Each tier may meet specified criteria regarding expected compression ratios achieved for a specified portion or number of data sets. Compression algorithms of each tier may be implemented by a different hardware device that may include hardware accelerators for the algorithms of the tier. Different tiers, and thus different hardware devices, achieve different levels of compression. A recommendation may be provided using compression information collected, such as from one of the hosts, regarding which hardware device to use for compression. The recommendation may be to purchase a license to use or whether to purchase a particular hardware device for compression. Compression information may be collected by a host that issues tagged I/Os providing a hint regarding what compression algorithm to use for the particular I/O operation data.

Abstract: Tiers of compression algorithms may be determined using compression information collected regarding compression ratios achieved for data sets using compression algorithms. Each tier may meet specified criteria regarding expected compression ratios achieved for a specified portion or number of data sets. Compression algorithms of each tier may be implemented by a different hardware device that may include hardware accelerators for the algorithms of the tier. Different tiers, and thus different hardware devices, achieve different levels of compression. A recommendation may be provided using compression information collected, such as from one of the hosts, regarding which hardware device to use for compression. The recommendation may be to purchase a license to use or whether to purchase a particular hardware device for compression. Compression information may be collected by a host that issues tagged I/Os providing a hint regarding what compression algorithm to use for the particular I/O operation data.

Abstract: Secure access to data on a storage system via direct connection to an internal fabric of the storage system may be provided. A storage system interface (SSI) may validate each I/O communication originating on the host system before allowing a corresponding I/O communication to be transmitted on the internal fabric. The validation may include applying predefined rules and/or ensuring that the I/O communication conforms to one or more technologies, e.g., NVMe. The SSI may be configured to encrypt I/O communications originating on a host system and to decrypt I/O communications received from the storage system, for example, in embodiments in which data is encrypted in flight from the host system to physical storage devices, and data may be encrypted at rest in memory of the storage system and/or on physical storage devices.

Abstract: A hardware offload includes a hash engine that performs hashing for a block-based storage system. The hash engine calculates multiple hash values for each input buffer provided by the storage system. The hash values may be calculated with variably offset and overlapping portions of the input buffer, wherein each portion is larger than the native block size of the storage system. The hardware offload may also include a compression engine that performs compression on the input buffer using the entire input buffer and/or chunks as compression domains.

Abstract: Data compression is performed on a storage system for which one or more host systems have direct access to data on the storage system. The storage system may compress the data for one or more logical storage units (LSUs) having data stored thereon, and may update compression metadata associated with the LSUs and/or the data portions thereof to reflect that the data is compressed. In response to a read request for a data portion received from a host application executing on the host system, compression metadata for the data portion may be accessed. If it is determined from the compression metadata that the data portion is compressed, the data compression metadata for the data portion may be further analyzed to determine how to decompress the data portion. The data portion may be retrieved and decompressed, and the decompressed data may be returned to the requesting application.

Abstract: A host system is connected to the internal fabric of a storage system without an intervening external network or director or other component of the storage system controlling the host system's access to the internal fabric. The host system may exchange I/O communications with physical storage devices and/or global memory over an I/O path that does not include any directors, for example, over the internal fabric to which the host system is directly attached. In embodiments in which at least a portion of the global memory is considered part of a director, the host system may be configured to communicate with such global memory over the internal fabric and without use of director compute resources.

Abstract: A storage system includes four storage engines, each storage engine including two compute nodes. Eight point-to-point connections are used to interconnect pairs of compute nodes on different storage engines, such that each compute node is connected to exactly two other compute nodes of the storage system. Atomic operations can be initiated by any compute node on any other compute node. Atomic operations received by a compute node on one of the point-to-point connections will be forwarded on the other point-to-point connection if the atomic operation is not directed to the compute node. During normal operation, atomic operations on a given compute node are performed on a host adapter associated with the compute node. Upon failure of the host adapter associated with the compute node, atomic operations may be performed on the compute node using the host adapter of the other compute node of the storage engine.

Abstract: Data compression is performed on a storage system for which one or more host systems have direct access to data on the storage system. The storage system may compress the data for one or more logical storage units (LSUs) having data stored thereon, and may update compression metadata associated with the LSUs and/or the data portions thereof to reflect that the data is compressed. In response to a read request for a data portion received from a host application executing on the host system, compression metadata for the data portion may be accessed. If it is determined from the compression metadata that the data portion is compressed, the data compression metadata for the data portion may be further analyzed to determine how to decompress the data portion. The data portion may be retrieved and decompressed, and the decompressed data may be returned to the requesting application.

Abstract: A storage system may assign a different encryption key to each logical storage unit (LSU) of a storage system. For each LSU, the encryption key of the LSU may be shared only with host systems authorized to access data of the LSU. In response to a read request for a data portion received from a host application executing on the host system, encryption metadata for the data portion may be accessed. If it is determined from the encryption metadata that the data portion is encrypted, the data encryption metadata for the data portion may be further analyzed to determine the encryption key for the data portion. The data may be retrieved from the storage system, for example, by performance of a direct read operation. The retrieved data may be decrypted, and the decrypted data may be returned to the requesting application.

Abstract: A storage system includes four storage engines, each storage engine including two compute nodes. Eight point-to-point connections are used to interconnect pairs of compute nodes on different storage engines, such that each compute node is connected to exactly two other compute nodes of the storage system. Atomic operations can be initiated by any compute node on any other compute node. Atomic operations received by a compute node on one of the point-to-point connections will be forwarded on the other point-to-point connection if the atomic operation is not directed to the compute node. During normal operation, atomic operations on a given compute node are performed on a host adapter associated with the compute node. Upon failure of the host adapter associated with the compute node, atomic operations may be performed on the compute node using the host adapter of the other compute node of the storage engine.

Abstract: A storage system includes four storage engines, each storage engine including two compute nodes. Eight point-to-point connections are used to interconnect pairs of compute nodes on different storage engines, such that each compute node is connected to exactly two other compute nodes of the storage system. Atomic operations can be initiated by any compute node on any other compute node. Atomic operations received by a compute node on one of the point-to-point connections will be forwarded on the other point-to-point connection if the atomic operation is not directed to the compute node. During normal operation, atomic operations on a given compute node are performed on a host adapter associated with the compute node. Upon failure of the host adapter associated with the compute node, atomic operations may be performed on the compute node using the host adapter of the other compute node of the storage engine.

Abstract: A hardware offload includes a hash engine that performs hashing for a block-based storage system. The hash engine calculates multiple hash values for each input buffer provided by the storage system. The hash values may be calculated with variably offset and overlapping portions of the input buffer, wherein each portion is larger than the native block size of the storage system. The hardware offload may also include a compression engine that performs compression on the input buffer using the entire input buffer and/or chunks as compression domains.

Abstract: Tiers of compression algorithms may be determined using compression information collected regarding compression ratios achieved for data sets using compression algorithms. Each tier may meet specified criteria regarding expected compression ratios achieved for a specified portion or number of data sets. Compression algorithms of each tier may be implemented by a different hardware device that may include hardware accelerators for the algorithms of the tier. Different tiers, and thus different hardware devices, achieve different levels of compression. A recommendation may be provided using compression information collected, such as from one of the hosts, regarding which hardware device to use for compression. The recommendation may be to purchase a license to use or whether to purchase a particular hardware device for compression. Compression information may be collected by a host that issues tagged I/Os providing a hint regarding what compression algorithm to use for the particular I/O operation data.

Abstract: A storage system includes a storage engine having a first compute node, a second compute node, a first fabric adapter, and a second fabric adapter, the first compute node having a first memory and the second compute node having a second memory. The first compute node is connected to both the first and second fabric adapters, and the second compute node is connected to both the second and first fabric adapters. Both fabric adapters are configured to perform atomic operations on a memory of its respective compute node, and each fabric adapter contains a multi-initiating module configured to enable both the first compute node and the second compute node to initiate memory access operations on its respective memory.

Abstract: Secure access to data on a storage system via direct connection to an internal fabric of the storage system may be provided. A storage system interface (SSI) may validate each I/O communication originating on the host system before allowing a corresponding I/O communication to be transmitted on the internal fabric. The validation may include applying predefined rules and/or ensuring that the I/O communication conforms to one or more technologies, e.g., NVMe. The SSI may be configured to encrypt I/O communications originating on a host system and to decrypt I/O communications received from the storage system, for example, in embodiments in which data is encrypted in flight from the host system to physical storage devices, and data may be encrypted at rest in memory of the storage system and/or on physical storage devices.

Abstract: A host system is connected to the internal fabric of a storage system without an intervening external network or director or other component of the storage system controlling the host system's access to the internal fabric. The host system may exchange I/O communications with physical storage devices and/or global memory over an I/O path that does not include any directors, for example, over the internal fabric to which the host system is directly attached. In embodiments in which at least a portion of the global memory is considered part of a director, the host system may be configured to communicate with such global memory over the internal fabric and without use of director compute resources.

Abstract: A host system may include metadata mapping logical storage devices and logical addresses therein to physical storage devices and physical addresses therein. For a read operation, the host system, if it is determined that the data is not in cache on the storage system, the host system may determine, from the device-mapping metadata, the physical storage device and physical location (e.g., address range) therein of the data to be read. The data then may be read from the physical storage device over the internal fabric of the storage system without use of a director. Data may be read from the physical storage device to the host system using RDMA communications that do not involve use of any CPU resources on the host system or the storage system.

Abstract: A storage system interface (SSI) located externally to a data storage system serves as an interface between a host system and the data storage system. The SSI may be part of the host system, and in some embodiments may be a separate and discrete component from the remainder of the host system, physically connected to the remainder of the host system by one or more buses that connect periphery devices to the remainder of the host system. The SSI may be physically connected directly to the internal fabric of the data storage system, and may be implemented on a card or chipset physically connected to the remainder of a host system by a PCIe bus. The SSI may provide functionality traditionally provided on data storage systems, enabling at least some I/O processing to be offloaded from data storage systems to hosts that include SSIs.