Abstract: Disclosed is a system for generating parity information for a data file in a distributed data structure system. Data objects in the data file are distributed into data buckets located in memory areas in servers interconnected by a network. An nth set of bucket group numbers are generated. A data bucket and a parity bucket are associated with a bucket group number in the nth set. Parity data for the data objects is generated and stored in a parity bucket associated with a bucket group number in the nth set. After adding a data object to the data file an additional data bucket may be provided for additional data object storage space. After adding a data bucket, a determination is made as to whether bucket availability has decreased below a predetermined threshold. If so, an (n+1)th set of bucket group numbers is generated and parity data for at least one of the data objects is stored in a parity bucket associated with a bucket group number in the (n+1)th set.

Abstract: A system and method for updating data. A first processing unit, such as an adaptor, receives a data update to a data block in a first storage device, such as a hard disk drive. Parity data for the data block is maintained in a second storage device, e.g., another hard disk drive. A parity group is comprised of the data block and corresponding parity data. The first processing unit determines whether the first processing unit controls access to the parity group, i.e., the state of lock ownership. The first processing unit transmits the data update to a second processing unit after determining that the first processing unit does not control access to the parity group. The second processing unit, another adaptor, performs an update after receiving the data update from the first processing unit. The first processing unit performs an update after determining that the first processing unit controls access to the parity group.

Abstract: Disclosed is a system for recovering data objects stored in a data bucket in a computer system comprised of servers interconnected by a network. Each server includes a storage area. Data buckets are included in a plurality of the storage areas throughout the network. A failure of a data bucket and nth bucket group number associated with the failed data bucket are determined, wherein n is an integer value greater than zero. Parity data in a parity bucket associated with the nth bucket group number is then located. The data objects in the failed data bucket are then recovered from the parity data in the located parity bucket.

Abstract: To satisfy host requests, two competing processors self-manage access to a shared resource. Each processor maintains a lock table listing that processor's access state regarding the shared resource. Each processor repeatedly sends the other processor a state announcement message representing the processor's state. These include birth cry, heartbeat, or death knell messages. Whenever certain join-entry conditions apply to a processor, the processor performs a JOIN operation to evaluate and attempt to synchronize its lock table with the other processor's lock table. If data is needed from the other processor's lock table, the processor changes its state announcement message to birth cry. If the processor successfully synchronizes its lock table with the other processor, the JOIN operation dictates DUAL mode for that processor. If the synchronization attempt fails, SOLO results. When JOIN completes, the processor operates in its chosen mode.

Abstract: A procedure controls execution of priority ordered tasks in a multi-nodel data processing system. The data processing system includes a node with a software-controlled processor and a hardware-configured queue-controller. The queue-controller includes a plurality of priority-ordered queues, each queue listing tasks having an assigned priority equal to a priority order assigned to the queue.

Abstract: In a data processing system having a plurality of queues for prioritizing I/O requests to a storage device, the priority of the queues for servicing is dynamically adjustable as opposed to each queue having a fixed priority. Dynamically adjusting the priority of a queue allows the priority of a queue with "stuck" requests to be raised so the "stuck" requests can be serviced very quickly.

Abstract: An information storage system selects target segments for garbage collection only if their age in the information storage system exceeds an age threshold value and, once past the age threshold, in the order of least utilized segments first. The system determines the age of a segment by determining the amount of time a segment has been located in direct access storage devices (DASD) of the information storage system and considers a segment for garbage collection only after the segment has been located in DASD for the selected age threshold value. From the set of candidate segments, the system chooses one or more for garbage collection in the order in which they will yield the most free space. The free space yield is determined by utilization data, so that the least utilized segments are garbage-collected first.

Abstract: A method enables a host processor, which employs variable length (VL) records, to transparently communicate with disk storage which employs fixed length (FL) sectors for storage of the VL records.

Abstract: A method enables a host processor, which employs variable length (VL) records, to communicate with disk storage which employs fixed length (FL) sectors for storage of the VL records.

Abstract: An array controller of a log structured array dynamically defines segments as data is taken from a write buffer and recorded into disk storage units of the log structured array such that a segment includes blocks selected from the various disk storage units at the time of segment definition so as to minimize disk arm travel. Each time a segment from the write buffer must be written to disk, the new disk segment is defined in response to current disk arm position for each disk platter in a disk storage unit. The array controller maintains a segment definition table that indicates the disk storage unit locations corresponding to each segment. In addition, garbage collection of used blocks is performed in response to disk arm idle time such that garbage collection is not performed only when there are too few empty segments. Rather, garbage collection is performed when a disk arm has been idle for a predetermined time. A variety of garbage collection schemes are described.

Abstract: A system and method are provided for managing persistent and non-persistent data in a log structured disk array. In a log structured array, data are first collected in a log stored in a buffer. When the buffer is full, the data are written to a disk drive. Persistent data are continually accessed over a long period of time such that, most disk activity is associated with persistent data. Preferably, the persistent data are stored in the part of the disk where access times are minimal (active region), such as in the middle cylinders of a disk. When writing data (including during garbage collection procedure) persistent data is written to the active region and non-persistent data is written to the other regions (inactive region). The persistence of the data is monitored. Persistent data is migrated to the active region and non-persistent is migrated to the inactive region.

Abstract: An apparatus for dynamically allocating memory includes a processor, a free buffer pool memory and a control memory which stores control block data structures. The control block data structures enable a segmentation of the free buffer pool memory into a series of free buffer pools, each free buffer pool comprising plural identical size buffers, each succeeding free buffer pool including a larger buffer size than a preceding free buffer pool. A selection size parameter for a given free buffer pool is a value that is larger than the buffer size comprising the given free buffer pool, but less than a next larger buffer size in the next of the series of free buffer pools.

Abstract: An efficient method for adding one or more new direct access storage devices (DASDs) to a redundant array of independent disks (RAID) array without disrupting ongoing operation of the array. In one embodiment, after contents of new DASDs are initialized off-line, the "parity" or "data" status of specific storage locations in the array is re-mapped, without any changes to the storage locations' contents. In a second embodiment, RAID arrays are efficiently expanded and parallelism is facilitated for original as well as new data by re-mapping parity and data tracks, changing the contents of a minimum number of tracks and maintaining the contents of others.

Abstract: A system for managing data storage devices of a data storage subsystem. A data storage system includes a controller coupled to multiple data storage devices. In response to a request, the controller allocates the devices' storage space into a number of storage partitions or "virtual devices." The request specifies the size and function mode for each storage partition. Each storage partition, having the requested size, is operated according to its requested function mode. This involves mapping access commands, which specify virtual addresses, to the proper physical addresses of the appropriate data storage devices.

Abstract: A log-structured array (LSA) includes a relatively large, non-volatile cache memory as well as a memory segment write buffer. The LSA cache memory contains both updated logical tracks received from the host system and also clean logical tracks read from direct access storage devices of the array. When the fraction of updated tracks in the cache memory exceeds a threshold value, updated tracks are moved from the LSA cache memory to the memory segment. With each modified track to be moved, adjacent modified tracks also are moved. The updated tracks are moved regardless of their location in the updated LRU list. Clean tracks in the LSA cache memory are moved into the memory segment as well, when additional clean tracks are needed in the cache memory. The clean tracks are selected from the bottom of the LSA cache clean track LRU list. Checking for adjacency also can occur at garbage collection time, performed in a background mode.

Abstract: In a data processing system employing a disk array, prediction of a possible failure of a disk drive initiates copying of the data away from the potentially failing disk drive to a spare disk drive before the failing drive actually fails. If the disk drive does fail before the copying of the contents to a spare disk drive is completed, rebuilding of the remaining contents within the failing disk drive is performed.

Abstract: A distributed data processing system includes a plurality of nodes interconnected by bidirectional communication links. Each node includes a control message line for handling of control messages and a control memory for storing the control messages. Each node further includes data message line for handling of data messages and a data memory for storing the data messages. A processor in the node causes the data message line to queue and dispatch data messages from the data memory and the control message line to queue and dispatch control messages from the control memory. Each node includes N bidirectional communication links enabling the node to have at least twice as much input/output bandwidth as the control message line and data message line, combined. An input/output switch includes a routing processor and is coupled between the N bidirectional communication links, the data message line and control message line.

Abstract: A method and means are disclosed in a Log Structure Array (LSA) storage subsystem for managing said subsystem without a need for an access to a complete LSA directory in a RAM. This object is achieved by maintaining (1) a subset of the LSA directory (referred to as LSA sub-directory) in a RAM where the LSA sub-directory comprises the logical track address of a predetermined number of most recently accessed logical tracks; (2) a journal of changes to the LSA directory which is maintained on a different power boundary than the LSA directory power boundary; and, (3) an array of bit maps, one bit map per segment which is used for fast garbage collection thus eliminating the need for having an access to a complete LSA directory in a RAM during garbage collection.

Abstract: A constant size storage can be managed to preserve locality of referencing where it is partitioned into linear addressable storage space for compressed symbol strings and a linked list addressable space for overflowing portions of each compressed string, a token to the overflow being embedded in the linear address. The linear space is readjusted periodically in a direction so as to maintain the amount of available overflow within to lie within a certain range of current usage. Changes in compression statistics result in changing overflow usage requiring readjustment to minimize internal fragmentation etc.

Abstract: A method and apparatus for sharing distributed spares of a first array with a second array in a storage subsystem which included a number of storage arrays requires that at least a first array of the subsystem has available distributed spares, and that at least a second array has a same number of "rows" as the first array, for spare sharing. When a device failure occurs on the second array, the data and parity blocks of the failed device are reconstructed and placed in the spares of the first array, and the block addresses are appropriately remapped to the spares. Distributed spare sharing is possible because the first array is configured to include at least one spare region/row, and the second array will never lose a number of blocks greater than the common number of rows in the first array in a single device failure.