Abstract: Data storage systems and methods for storing data are described herein. The storage system may be integrated with or coupled with a compute cluster or super computer having multiple computing nodes. A plurality of nonvolatile memory units may be included with computing nodes, coupled with computing nodes or coupled with input/output nodes. The input/output nodes may be included with the compute cluster or super computer, or coupled thereto. The nonvolatile memory units store data items provided by the computing nodes, and the input/output nodes maintain where the data items are stored in the nonvolatile memory units via a hash table distributed among the input/output nodes. The use of a distributed hash table allows for quick access to data items stored in the nonvolatile memory units even as the computing nodes are writing large amounts of data to the storage system quickly in bursts.

Abstract: Data storage systems and methods for storing data are described herein. The storage system may be integrated with or coupled with a compute cluster or super computer having multiple computing nodes. A plurality of nonvolatile memory units may be included with computing nodes, coupled with computing nodes or coupled with input/output nodes. The input/output nodes may be included with the compute cluster or super computer, or coupled thereto. The nonvolatile memory units store data items provided by the computing nodes, and the input/output nodes maintain where the data items are stored in the nonvolatile memory units via a hash table distributed among the input/output nodes. The use of a distributed hash table allows for quick access to data items stored in the nonvolatile memory units even as the computing nodes are writing large amounts of data to the storage system quickly in bursts.

Abstract: Data storage systems and methods for storing data are described herein. The storage system includes at least two data storage nodes for storing portions of a distributed hash table and related data. After a first node attempts to complete a write request at a second node and is unable to complete the request, the first node ceases responses to interactions from other nodes. Once the first node's failure to respond has caused a sufficient number of nodes to cease responding, the nodes enter a service mode to resolve the live lock. While in live lock, the nodes determine the oldest, unfulfilled request using a system-wide logical timestamp associated with write requests. Once the oldest request is determined, a removal vote to remove the non-responsive node from the group is initiated and, if other nodes agree, the non-responsive node is removed from the group of nodes.

Abstract: Data storage systems and methods for storing data are described herein. The storage system may be integrated with or coupled with a compute cluster or super computer having multiple computing nodes. A plurality of nonvolatile memory units may be included with computing nodes, coupled with computing nodes or coupled with input/output nodes. The input/output nodes may be included with the compute cluster or super computer, or coupled thereto. The nonvolatile memory units store data items provided by the computing nodes, and the input/output nodes maintain where the data items are stored in the nonvolatile memory units via a hash table distributed among the input/output nodes. The use of a distributed hash table allows for quick access to data items stored in the nonvolatile memory units even as the computing nodes are writing large amounts of data to the storage system quickly in bursts.

Abstract: A system for data migration between high performance computing architectures and data storage disks includes an intermediate data migration handling system which has an intermediate data storage module coupled to the computer architecture to store data received, and a data controller module which includes data management software supporting the data transfer activity between the intermediate data storage module and the disk drives in an orderly manner independent of the random I/O activity of the computer architecture. RAID calculations are performed on the data prior to storage in the intermediate storage module, as well as when reading data from it for assuring data integrity, and carrying out reconstruction of corrupted data. The data transfer to the disk drives is actuated in sequence determined by the data management software based on minimization of seeking time, tier usage, predetermined time since the previous I/O cycle, or fullness of the intermediate data storage module.

Abstract: Data storage systems and methods for storing data are described herein. The storage system may be integrated with or coupled with a compute cluster or super computer having multiple computing nodes. A plurality of nonvolatile memory units may be included with computing nodes, coupled with computing nodes or coupled with input/output nodes. The input/output nodes may be included with the compute cluster or super computer, or coupled thereto. The nonvolatile memory units store data items provided by the computing nodes, and the input/output nodes maintain where the data items are stored in the nonvolatile memory units via a hash table distributed among the input/output nodes. The use of a distributed hash table allows for quick access to data items stored in the nonvolatile memory units even as the computing nodes are writing large amounts of data to the storage system quickly in bursts.

Abstract: Data storage systems and methods for storing data are described herein. The storage system includes at least two data storage nodes for storing portions of a distributed hash table and related data. After a first node attempts to complete a write request at a second node and is unable to complete the request, the first node ceases responses to interactions from other nodes. Once the first node's failure to respond has caused a sufficient number of nodes to cease responding, the nodes enter a service mode to resolve the live lock. While in live lock, the nodes determine the oldest, unfulfilled request using a system-wide logical timestamp associated with write requests. Once the oldest request is determined, a removal vote to remove the non-responsive node from the group is initiated and, if other nodes agree, the non-responsive node is removed from the group of nodes.

Abstract: A latency reduction method for read operations of an array of N solid-state storage devices having n solid-state storage devices for data storage and p solid-state storage devices for storing parity data is provided. Utilizing the parity generation engine fault tolerance for a loss of valid data from at least two of the N solid-state storage devices, the integrity of the data is determined when N?1 of the solid-state storage devices have completed executing a read command. If the data is determined to be valid, the missing data of the Nth solid-state storage device is reconstructed and the data transmitted to the requesting processor. By that arrangement the time necessary for the Nth solid-state storage device to complete execution of the read command is saved, thereby improving the performance of the solid-state memory system.

Abstract: A method for auto-correction of errors in an array of solid-state storage devices having a plurality of storage channels dedicated to storing parity data to provide fault tolerance for a loss of at least two of the plurality of storage channels. A read operation from the storage channels transfers data to a plurality of channel memories. The data in the channel memories is checked to confirm the data is valid. Responsive to detection of invalid data, the data may be tested to identify the storage channel in error, including sequentially excluding data read form a different one of the plurality of channel memories from a parity check and determining the validity of data from remaining channel memories. If valid data is obtained, the storage channel from which the data was excluded is identified as the storage channel in error.

Abstract: A method for auto-correction of errors in an array of disk storage devices having a plurality of disk storage devices dedicated to storing parity data to provide fault tolerance for a loss of at least two of the plurality of disk storage devices. A read operation from the storage channels transfers data to a plurality of disk channel memories. The data in the disk channel memories is checked to confirm the data is valid. Responsive to detection of invalid data, the data may be tested to identify the disk storage channel in error, including sequentially excluding data read from a different one of the plurality of disk channel memories from a parity check and determining the validity of data from remaining disk channel memories. If valid data is obtained, the disk storage channel from which the data was excluded is identified as the disk storage channel in error.

Abstract: In order to detect problematic drives in random arrays of independent disks, the system measures the latency of executing command sets which are broadcast to all disks in the data storage system and the results are compared to identify which disks take substantially longer to complete the requests. Disks that take longer to complete requests are likely to be problematic and are candidates for further examination and replacement. The disks in each tier group are compared to determine if any disk in that group exhibits problems. Also, counters for each tier group are compared to determine if the problem is with the disk or with the channel of the tier group. The latency of each disk in the tier group is saved in a table to provide a histogram of the latency of the disks in the tier group. Histograms of the disks in a single tier group are compared to determine if a specific disk is problematic.

Abstract: In a data storage system, failed disk drives are switched temporarily off-line to be quickly rebuilt by executing a journaling/rebuild algorithm which tracks the updates to the failed disk drive into a journal structure created in a non-volatile memory. The journal information is used to update those data sections of the disk drive affected by updates after the disk drive is failed. The journal information is stored in bit maps indicating which portions of the disk drive have been updated with new data while the disk was failed. As an option, the system permits verification of data consistency on the data section of the disk drive which have not been affected by the updates. The journaling/rebuild of failed disks is applicable, among others, to RAID data storage systems.

Abstract: In a data storage system, failed disk drives are switched temporarily off-line to be quickly rebuilt by executing a journaling/rebuild algorithm which tracks the updates to the failed disk drive into a journal structure created in a non-volatile memory. The journal information is used to update those data sections of the disk drive affected by updates after the disk drive is failed. The journal information is stored in bit maps indicating which portions of the disk drive have been updated with new data while the disk was failed. As an option, the system permits verification of data consistency on the data section of the disk drive which have not been affected by the updates. The journaling/rebuild of failed disks is applicable, among others, to RAID data storage systems.

Abstract: In order to detect problematic drives in random arrays of independent disks, the system measures the latency of executing command sets which are broadcast to all disks in the data storage system and the results are compared to identify which disks take substantially longer to complete the requests. Disks that take longer to complete requests are likely to be problematic and are candidates for further examination and replacement. The disks in each tier group are compared to determine if any disk in that group exhibits problems. Also, counters for each tier group are compared to determine if the problem is with the disk or with the channel of the tier group. The latency of each disk in the tier group is saved in a table to provide a histogram of the latency of the disks in the tier group. Histograms of the disks in a single tier group are compared to determine if a specific disk is problematic.

Abstract: A method for auto-correction of errors in an array of disk storage devices (210) having a plurality of disk storage devices (210I, 210J) dedicated to storing parity data to provide fault tolerance for a loss of at least two of the plurality of disk storage devices (210A-210J). A read operation from the storage channels (210A-210J) transfers data to a plurality of disk channel memories (220A-220J). The data in the disk channel memories (220A-220J) is checked to confirm the data is valid. Responsive to detection of invalid data, the data may be tested to identify the disk storage channel in error, including sequentially excluding data read from a different one of the plurality of disk channel memories (220A-220J) from a parity check and determining the validity of data from remaining disk channel memories. If valid data is obtained, the disk storage channel from which the data was excluded is identified as the disk storage channel in error.

Abstract: A performance optimized RAID Level 3 storage access controller with a unique XOR engine placement at the host/network side of the cache. The invention utilizes multiple data communications channels and a centralized cache memory in conjunction with this unique XOR placement to maximize performance and fault tolerance between a host network and data storage. Positioning the XOR engine at the host/network side of the cache allows the storage devices to be fully independent. Since the XOR engine is placed in the data path and the parity is generated in real-time during cache write transfers, the bandwidth overhead is reduced to zero. For high performance RAID controller applications, a system architecture with minimal bandwidth overhead provides superior performance.

Abstract: A performance optimized RAID Level 3 storage access controller with a unique XOR engine placement at the host/network side of the cache. The invention utilizes multiple data communications channels and a centralized cache memory in conjunction with this unique XOR placement to maximize performance and fault tolerance between a host network and data storage. Positioning the XOR engine at the host/network side of the cache allows the storage devices to be fully independent. Since the XOR engine is placed in the data path and the parity is generated in real-time during cache write transfers, the bandwidth overhead is reduced to zero. For high performance RAID controller applications, a system architecture with minimal bandwidth overhead provides superior performance.

Abstract: A performance optimized RAID Level 3 storage access controller with a unique XOR engine placement at the host/network side of the cache. The invention utilizes multiple data communications channels and a centralized cache memory in conjunction with this unique XOR placement to maximize performance and fault tolerance between a host network and data storage. Positioning the XOR engine at the host/network side of the cache allows the storage devices to be fully independent. Since the XOR engine is placed in the data path and the parity is generated in real-time during cache write transfers, the bandwidth overhead is reduced to zero. For high performance RAID controller applications, a system architecture with minimal bandwidth overhead provides superior performance.