Abstract: A controller including an object aggregator process that combines multiple data objects into a data segment, and transfers the data segment with reduced location metadata to storage media of at least one of multiple storage units. An erasure coder process generates code to encode the data segment into an erasure code that protects against concurrent data loss in the multiple storage units based on data reconstruction using a first responder, a second responder and a last responder.

Abstract: A method for distributed coding in a storage array is presented. The method includes dividing data into multiple stripes for storage in a storage array including storage devices with a topology of a hypercube of a dimension t?3. The storage devices in same hypercubes of dimension t?1 including the hypercube of a dimension t have even parity. Global parities are added to the hypercube such that a minimum distance of a code is enhanced.

Abstract: A method for hierarchical correction coding includes converting data for a storage system into w storage device arrays, each storage device array including n storage devices, and each storage device divided into m sectors or pages. The n storage devices are grouped into l groups of t storage devices each. Erasures in the w storage device arrays are corrected based on protecting each row and column in each m×n array by an erasure-correcting code. Each group of t storage devices contains extra parities to correct extra erasures in addition to erasures corrected by vertical parities in each m×t subarray, and w, n, m, l and t are positive integers.

Abstract: Embodiments relate to dynamically selecting an erasure code. State data is tracked to ascertain frequency of file access. One of at least two erasure codes are selected based on the tracked state data in order to lower data recovery cost. The erasure code may be selected as either a product code or a local reconstruction code. Each erasure code includes a mode that is either a fast code or a compact code. The fast code features a low recovery cost and the compact code features a low storage overhead for less frequently accessed data. Data is encoded with one of the selected erasure codes and one of the modes of the selected erasure code. Data blocks are dynamically converted between the fast and compact codes of the selected erasure code responsive to a workload change.

Abstract: A data storage structure, comprising: a plurality of storage units, each comprising: a storage media; and a library executive configured to manage the storage media. The structure further comprises a buffer connected to a controller, the controller comprising: a host interface configured to receive the instruction from the host machine; an object aggregator configured to combine the plurality of data objects into a data segment; a persistent write buffer configured to store the data segment; a persistent map configured to identify a location of each of the plurality of objects in the data segment; an erasure coder configured to encode the data segment into an erasure code; a destager configured to transfer the data segment from the persistent write buffer to the storage media in a given storage unit; and a library controller configured to communicate with the library executive in the given storage unit.

Abstract: Embodiments relate to correcting erasures in a storage array. An aspect includes dividing data into a plurality of stripes for storage in a storage array comprising a plurality of storage locations, each stripe comprising M rows and N columns, each of the M rows including a number r of row parities, wherein r is greater than zero. Another aspect includes dividing each stripe into two or more column sets, each column set comprising a respective set of one or more columns of the stripe. Another aspect includes adding a respective first responder parity to each column set, wherein each first responder parity gives parity information for only the two or more columns in the first responder parity's respective column set.

Abstract: Embodiments relate to correcting erasures in a storage array. An aspect includes dividing data into a plurality of stripes for storage in a storage array comprising a plurality of storage locations, each stripe comprising M rows and N columns, each of the M rows including a number r of row parities, wherein r is greater than zero. Another aspect includes dividing each stripe into two or more column sets, each column set comprising a respective set of one or more columns of the stripe. Another aspect includes adding a respective first responder parity to each column set, wherein each first responder parity gives parity information for only the two or more columns in the first responder parity's respective column set. Yet another aspect includes, based on an isolated erasure in a column set, correcting the isolated erasure by reading data from only storage locations corresponding to the column set in which the isolated erasure occurred using the first responder parity of the column set.

Abstract: A data storage structure, comprising: a plurality of storage units, each comprising: a storage media; and a library executive configured to manage the storage media. The structure further comprises a buffer connected to a controller, the controller comprising: a host interface configured to receive the instruction from the host machine; an object aggregator configured to combine the plurality of data objects into a data segment; a persistent write buffer configured to store the data segment; a persistent map configured to identify a location of each of the plurality of objects in the data segment; an erasure coder configured to encode the data segment into an erasure code; a destager configured to transfer the data segment from the persistent write buffer to the storage media in a given storage unit; and a library controller configured to communicate with the library executive in the given storage unit.

Abstract: A method for hierarchical correction coding includes converting data for a storage system into w storage device arrays, each storage device array including n storage devices, and each storage device divided into m sectors or pages. The n storage devices are grouped into l groups of t storage devices each. Erasures in the w storage device arrays are corrected based on protecting each row and column in each m×n array by an erasure-correcting code. Each group of t storage devices contains extra parities to correct extra erasures in addition to erasures corrected by vertical parities in each m×t subarray, and w, n, m, l and t are positive integers.

Abstract: A method for distributed coding in a storage array is presented. The method includes dividing data into multiple stripes for storage in a storage array including storage devices with a topology of a hypercube of a dimension t?3. The storage devices in same hypercubes of dimension t?1 including the hypercube of a dimension t have even parity. Global parities are added to the hypercube such that a minimum distance of a code is enhanced.

Abstract: Bitmap compression for fast searches and updates is provided. Compressing a bitmap includes receiving a bitmap to compress, and reading the bitmap to determine a value of a bit location for all bits in the bitmap. In one embodiment, a compressed bitmap is created by encoding a variable number of bytes to represent a distance between adjacent 1s in the uncompressed bitmap. In another embodiment, a compressed bitmap is created by representing a distance between adjacent 1s in the uncompressed bitmap using a plurality of bits, and encoding a marker word to indicate the number of bits used to represent the distance.

Abstract: A computer-implemented method is provided for increasing the failure tolerance of an array of storage elements in a storage system. The computer-implemented method includes configuring an array to include a plurality of storage elements in n>1 sets of storage elements. The computer-implemented method also includes configuring an erasure-correcting code such that at least one column of the storage elements of the array stores row parity information, and at least one row of the storage elements of the array stores column parity information. Still yet, the computer-implemented method includes, subsequent to a failure of one of the storage elements storing data, selecting a recipient storage element from the array, and rebuilding at least a portion of the data onto the recipient storage element by performing a parity exchange operation that retains a failure tolerance of the set of storage elements containing the failed storage element.

Abstract: Embodiments relate to dynamically selecting an erasure code. State data is tracked to ascertain frequency of file access. One of at least two erasure codes are selected based on the tracked state data in order to lower data recovery cost. The erasure code may be selected as either a product code or a local reconstruction code. Each erasure code includes a mode that is either a fast code or a compact code. The fast code features a low recovery cost and the compact code features a low storage overhead for less frequently accessed data. Data is encoded with one of the selected erasure codes and one of the modes of the selected erasure code. Data blocks are dynamically converted between the fast and compact codes of the selected erasure code responsive to a workload change.

Abstract: Embodiments relate to a system with multiple erasure codes, and selecting and encoding for a write file with one of the codes to mitigate costs associated with storage recovery. The codes include a fast recovery code for frequently accessed data and a higher storage efficiency code for less frequently accessed data. State data is tracked to ascertain frequency of access to the file. One of the erasure codes is dynamically selected based on the tracked data, with the focus of the code select to lower recovery costs, and the data is encoded with the selected erasure code. Accordingly, the original coding of the write file is subject to change based on the tracked state data.

Abstract: Embodiments relate to correcting erasures in a storage array. An aspect includes dividing data into a plurality of stripes for storage in a storage array comprising a plurality of storage locations, each stripe comprising M rows and N columns, each of the M rows including a number r of row parities, wherein r is greater than zero. Another aspect includes dividing each stripe into two or more column sets, each column set comprising a respective set of one or more columns of the stripe. Another aspect includes adding a respective first responder parity to each column set, wherein each first responder parity gives parity information for only the two or more columns in the first responder parity's respective column set. Yet another aspect includes, based on an isolated erasure in a column set, correcting the isolated erasure by reading data from only storage locations corresponding to the column set in which the isolated erasure occurred using the first responder parity of the column set.

Abstract: Embodiments relate to a system with multiple erasure codes, and selecting and encoding for a write file with one of the codes to mitigate costs associated with storage recovery. The codes include a fast recovery code for frequently accessed data and a higher storage efficiency code for less frequently accessed data. State data is tracked to ascertain frequency of access to the file. One of the erasure codes is dynamically selected based on the tracked data, with the focus of the code select to lower recovery costs, and the data is encoded with the selected erasure code. Accordingly, the original coding of the write file is subject to change based on the tracked state data.

Abstract: Embodiments relate to a computer system for storing data on a time multiplexed redundant array of independent tapes. An aspect includes a memory device that buffers data received by the computer system to be written to a set of tape data storage devices. The data is written to the set of tape data storage devices in blocks that form parity stripes across the set of tape data storage device. Aspects further includes a tape drive that writes data to one of the set of tape data storage devices at a time in a tape-sequential manner and a processor that computes a parity value for each of the parity stripes. The tape drive writes the parity values for each of the parity stripes to a last subset of tapes of the set of tape data storage devices.

Abstract: Embodiments include generating an error correction code by identifying two error-correcting codes of length n with minimum distances d and 2d, such that the second code is contained into the first code. Data is then encoded into t?1 blocks using the first code, and into the t-th block using the second block. The first t?1 encoded blocks are stored into t?1 blocks of memory locations, while the exclusive-OR (XOR) of such t?1 encoded blocks and the t-th encoded block is stored into a t-th block of memory location. The final encoded vector of length to is identified as belonging in a code with minimum distance 2d. Rotations of an encoded vector jn times are identified as belonging to the same equivalence class. A received vector or its rotations jn times can be successfully identified in the presence of up to d?1 errors, while d errors are identified as uncorrectable errors.

Abstract: Embodiments of the invention provide parity logs for raid systems with variable-capacity media. In one embodiment, a system includes a first set of data storage media devices having variable capacity. The storage devices include a data portion of a parity data set for storing write data being striped to the first. The system further includes a second set of data storage media devices having variable capacity. The second set includes a linear address space of blocks for storing a parity portion of the parity data set. The linear address space is written in a log form. The first and second sets comprise at least one array in a RAID configuration. The system writes the parity portion of the parity data set to the second set, which enables each storage device among the first set to be written to full capacity.

Abstract: Embodiments relate to a system with multiple erasure codes, and selecting and encoding for a write file with one of the codes to mitigate costs associated with storage recovery. The codes include a fast recovery code for frequently accessed data and a higher storage efficiency code for less frequently accessed data. State data is tracked to ascertain frequency of access to the file. One of the erasure codes is dynamically selected based on the tracked data, with the focus of the code select to lower recovery costs, and the data is encoded with the selected erasure code. Accordingly, the original coding of the write file is subject to change based on the tracked state data.