Abstract: Data storage systems and methods for storing data are described herein. An intermediate storage system is coupled with a High Performance Computing (HPC) system. A plurality of burst engines of the intermediate storage system are directly connected to compute nodes of the HPC. The burst engines are operative to receive file level data from the compute nodes at data transfer rates that at least match to the burst mode output rate of the compute node. The burst engines store the data to high bandwidth storage devices of the intermediate storage system at a data rate that at least matches the burst output rate of the compute node. During a compute phase of the compute node, data is replayed from the intermediate storage devices to a primary storage system at a lower data transfer rate.

Abstract: Systems and methods are provided for expanding the available memory of a storage controller. The systems and methods utilize a PCIe memory controller connected to the backend interface of the storage controller. Memory of the PCIe memory controller is memory mapped to controller memory of the storage controller. The PCIe connection allows the storage controller to access the memory of the PCIe memory controller with latencies similar to that of the controller memory.

Abstract: Data storage systems and methods for storing data are described herein. An intermediate storage system is coupled with a High Performance Computing (HPC) system. A plurality of burst engines of the intermediate storage system are directly connected to compute nodes of the HPC. The burst engines are operative to receive file level data from the compute nodes at data transfer rates that at least match to the burst mode output rate of the compute node. The burst engines store the data to high bandwidth storage devices of the intermediate storage system at a data rate that at least matches the burst output rate of the compute node. During a compute phase of the compute node, data is replayed from the intermediate storage devices to a primary storage system at a lower data transfer rate.

Abstract: Data storage systems and methods for storing data are described herein. An intermediate storage system is coupled with a High Performance Computing (HPC) system. A plurality of burst engines of the intermediate storage system are directly connected to compute nodes of the HPC. The burst engines are operative to receive file level data from the compute nodes at data transfer rates that at least match to the burst mode output rate of the compute node. The burst engines store the data to high bandwidth storage devices of the intermediate storage system at a data rate that at least matches the burst output rate of the compute node. During a compute phase of the compute node, data is replayed from the intermediate storage devices to a primary storage system at a lower data transfer rate.

Abstract: Provided herein are systems, apparatuses and methods (i.e., utilities) that allow for increasing the bandwidth of a processing complex of a storage controller. The utilities utilize a symmetrical approach where PCIe switches overcome limitations of prior art processor complexes. The symmetrical approach provided by the disclosed utilities as incorporated into a storage controller provides equal access from any host path/channel to any drive path/channel (i.e., storage element). More specifically, a first or a first set of PCIe switches connect front-end PCIe host bus adaptors, which are connectable to host systems, to front-end data paths of a plurality of PCIe memory controllers. A second or second set of PCIe switches connect backend host bus adapters, which are connectable to storage elements, to back-end data paths of the plurality of PCIe memory controllers. The symmetrical architecture provides at least twice the bandwidth of prior art architectures.

Abstract: Systems and methods are provided for expanding the available memory of a storage controller. The systems and methods utilize a PCIe memory controller connected to the backend interface of the storage controller. Memory of the PCIe memory controller is memory mapped to controller memory of the storage controller. The PCIe connection allows the storage controller to access the memory of the PCIe memory controller with latencies similar to that of the controller memory.

Abstract: Disclosed is a storage system. The storage system includes a redundant array of inexpensive disks (RAID) controller. The RAID controller includes a flash memory controller coupled to a flash memory. The flash memory controller may perform background management tasks. These include logging and error reporting, address translation, cache table management, bad block management, defect management, wear leveling, and garbage collection. The array controller also allows the flash memory to be divided into multiple mappings.

Abstract: System and methods for managing I/O write requests of host systems to physical storage. A storage subsystem includes a plurality of storage devices where each storage device is configured to provide data storage. At least a pair of redundant controllers is connected to the plurality of storage devices for executing the I/O write requests from the host systems. A received I/O write request is initially saved in a controller memory of one of the controllers and mirrored in controller memory of the other controller. In one embodiment, the I/O write request is transferred to a flash memory device for subsequent transfer to the storage devices. Once transferred to the flash memory device, the I/O write request may be flushed from the controller memories. The I/O write request may then be transferred to the storage devices from the flash memory device as a background operation.

Abstract: Methods and systems for integrated data management and block level storage management in a storage system having a controller with a multi-core processor. A hypervisor manages the multi-core processor such that at least one processor is used in a first virtual machine on which a block level storage management process is operable. The first virtual machine may also operate under the control of a real-time operating system. The hypervisor also defines a second virtual machine using one or more other processor cores and operates a data management application process optionally under control of a general purpose operating system. The optional general purpose operating system and the data management application process access storage devices by communicating through the hypervisor with the block level storage management process operable in the first virtual machine.

Abstract: Provided herein are systems, apparatuses and methods (i.e., utilities) that allow for increasing the bandwidth of a processing complex of a storage controller. The utilities utilize a symmetrical approach where PCIe switches overcome limitations of prior art processor complexes. The symmetrical approach provided by the disclosed utilities as incorporated into a storage controller provides equal access from any host path/channel to any drive path/channel (i.e., storage element). More specifically, a first or a first set of PCIe switches connect front-end PCIe host bus adaptors, which are connectable to host systems, to front-end data paths of a plurality of PCIe memory controllers. A second or second set of PCIe switches connect backend host bus adapters, which are connectable to storage elements, to back-end data paths of the plurality of PCIe memory controllers. The symmetrical architecture provides at least twice the bandwidth of prior art architectures.

Abstract: Disclosed is a storage system architecture. An Environmental service module (ESM) is coupled to one or more array controllers. The ESM is configured with a central processing unit and one or more assist functions. The assist functions may include nonvolatile memory. This nonvolatile memory may be used for write caching, mirroring data, and/or configuration data. The assist functions, or the ESM, may be controlled by the array controllers using SCSI or RDMA commands.

Abstract: Systems and associated methods for flexible scalability of storage systems. In one aspect, a storage controller may include an interface to a fabric adapted to permit each storage controller coupled to the fabric to directly access memory mapped components of all other storage controllers coupled to the fabric. The CPU and other master device circuits within a storage controller may directly address memory an I/O devices directly coupled thereto within the same storage controller and may use RDMA features to directly address memory an I/O devices of other storage controllers through the fabric interface.

Abstract: Data storage systems and methods for storing data are described herein. An intermediate storage system is coupled with a High Performance Computing (HPC) system. A plurality of burst engines of the intermediate storage system are directly connected to compute nodes of the HPC. The burst engines are operative to receive file level data from the compute nodes at data transfer rates that at least match to the burst mode output rate of the compute node. The burst engines store the data to high bandwidth storage devices of the intermediate storage system at a data rate that at least matches the burst output rate of the compute node. During a compute phase of the compute node, data is replayed from the intermediate storage devices to a primary storage system at a lower data transfer rate.

Abstract: Disclosed is a storage system. A network interface device (NIC) receives network storage commands from a host. The NIC may cache the data to/from the storage commands in a solid-state disk. The NIC may respond to future network storage command by supplying the data from the solid-state disk rather than initiating a network transaction.

Abstract: A method for selectively storing data identified by a software application in higher performance media may include executing control programming for an operating system and a software application hosted by the operating system. The software application assigns a first importance level to a first portion of data and a second importance level to a second portion of data. A first portion of data having the first importance level assigned by the software application is stored in a first storage medium at the instruction of the operating system. A second portion of data having the second importance level assigned by the software application is stored in a second storage medium at the instruction of the operating system. The second storage medium has at least one performance, reliability, or security characteristic different from the first storage medium.

Abstract: System and methods for managing I/O write requests of host systems to physical storage. A storage subsystem includes a plurality of storage devices where each storage device is configured to provide data storage. At least a pair of redundant controllers is connected to the plurality of storage devices for executing the I/O write requests from the host systems. A received I/O write request is initially saved in a controller memory of one of the controllers and mirrored in controller memory of the other controller. In one embodiment, the I/O write request is transferred to a flash memory device for subsequent transfer to the storage devices. Once transferred to the flash memory device, the I/O write request may be flushed from the controller memories. The I/O write request may then be transferred to the storage devices from the flash memory device as a background operation.

Abstract: Storage systems configured for improved N-way connectivity among all of a plurality of storage controllers and all of a plurality of storage devices in the system. All controllers of the storage system are coupled through a switched fabric communication medium to all of the storage devices of the storage system. Thus, the back-end interface of each storage controller of the storage system is used for all communications with any of the storage devices as well as for any communications among the controllers to coordinate the N-way distribution of stored data in a declustered RAID storage environment. This use of the back-end channel for all storage controller to storage device N-way connectivity as well as controller to controller N-way connectivity eliminates the need for a dedicated inter-controller interface for such N-way connectivity and eliminates the over-utilization of a front-end (e.g., network) communication path for providing N-way connectivity in the storage system.

Abstract: Methods and systems for integrated data management and block level storage management in a storage system having a controller with a multi-core processor. A hypervisor manages the multi-core processor such that at least one processor is used in a first virtual machine on which a block level storage management process is operable. The first virtual machine may also operate under the control of a real-time operating system. The hypervisor also defines a second virtual machine using one or more other processor cores and operates a data management application process optionally under control of a general purpose operating system. The optional general purpose operating system and the data management application process access storage devices by communicating through the hypervisor with the block level storage management process operable in the first virtual machine.

Abstract: Systems and associated methods for flexible scalability of storage systems. In one aspect, a storage controller may include an interface to a fabric adapted to permit each storage controller coupled to the fabric to directly access memory mapped components of all other storage controllers coupled to the fabric. The CPU and other master device circuits within a storage controller may directly address memory an I/O devices directly coupled thereto within the same storage controller and may use RDMA features to directly address memory an I/O devices of other storage controllers through the fabric interface.

Abstract: Disclosed is a storage system architecture. An Environmental service module (ESM) is coupled to one or more array controllers. The ESM is configured with a central processing unit and one or more assist functions. The assist functions may include nonvolatile memory. This nonvolatile memory may be used for write caching, mirroring data, and/or configuration data. The assist functions, or the ESM, may be controlled by the array controllers using SCSI or RDMA commands.