Abstract: A method for communicating between devices in a network includes creating an I/O tunnel between a first device and a second device through the network. The I/O tunnel is associated with I/O resources in both the first device and the second device, and wherein at least one of the I/O resources comprises a buffer resource. A data transfer operation may be sent between the first device and the second device by consuming at least some of the I/O resources associated with the I/O tunnel. A plurality of commands or a plurality of responses can be aggregated into a single buffer resource. Upon completion of the data transfer operation, the I/O resources that are consumed are automatically renewed.

Abstract: A method for communicating between devices in a network includes creating an I/O tunnel between a first device and a second device through the network. The I/O tunnel is associated with I/O resources in both the first device and the second device, and wherein at least one of the I/O resources comprises a buffer resource. A data transfer operation may be sent between the first device and the second device by consuming at least some of the I/O resources associated with the I/O tunnel. A plurality of commands or a plurality of responses can be aggregated into a single buffer resource. Upon completion of the data transfer operation, the I/O resources that are consumed are automatically renewed.

Abstract: A system for communicating between two devices in a network in which a semi-persistent tunnel is established between the two devices in advance of data communication. The semi-persistent tunnel includes resources that are pre-allocated in a first device at a first end of the communication link by a second device at the second end of the communication link. The first and second devices implement a plurality of processes for handling data transfer operations. Preferably, the semi-persistent tunnel also includes resources that are pre-allocated in a device at the second end of the communication link by the device at the first end of the communication link to allow bi-directional communication. Data transfer operations transmitted through the tunnel include an identification of specific resources of the pre-allocated resources that are to handle the data transfer operation.

Abstract: A data storage system adapted to maintain redundant data storage sets at a destination location(s) is disclosed. The data storage system establishes a copy set comprising a source volume and a destination volume. Data written to a source volume is automatically copied to the destination volume. The data storage system maintains a data log that may be activated when the destination volume is inaccessible due to, for example, a malfunction in the destination storage system or in the communication link between the source system and the destination system. I/O commands and the data associated with those commands are written to the data log, and after a destination system becomes available the information in the data log is merged into the destination volume to conform the data in the destination volume to the data in the source volume. The data log competes for disk capacity with other volumes on the system, and log memory is allocated as needed.

Abstract: A system in which data operations for a data replication group are received in-order, and buffered. When the operations are complete, they are stored in a non-volatile memory atomically with a group sequence number. The cache is preferably mirrored. This creates a persistent association between the data operation and the sequence number. After the atomic store is performed in at least one non-volatile cache, the data operation is propagated to another member of the data replication group along with the group sequence number. In the other member, the data operation is cached at least once in a non-volatile cache atomically with the group sequence number. In this manner, the set of group sequence numbers for a plurality of operations forms a continuous ordering of the data operations.

Abstract: A method is provided for storing and retrieving data in a network-attached data storage device by a cooperatively multitasking real time operating system configured to execute datapath routines and a general purpose operating system kernel configured to communicate with the network.

Abstract: Apparatus and method for efficiently arranging and searching data in a memory space, such as a cache memory of a data storage array controller. A data structure comprises a skip list of nodes having an array of forward pointers. Each node has a node level derived from an index at which the node is stored in a table in the memory space. The total available number of nodes is preferably selected to be less than half of the largest power of 2 that can be expressed by a number of bits of the index, and the nodes are preferably stored at only even or only odd indices of the table. In such case, a free list of nodes is preferably generated from an array of pairs of counts and indices to identify the available nodes. Additional table structures can further be provided to enhance data arrangement and searching functions.

Abstract: An atomic write data structure is provided comprising a stripe buffer list descriptor defining state information of a stripe buffer list entry associating a virtual stripe of data storage space with cache data. An associated method is provided comprising constructing metadata associating a virtual stripe of data storage space with cache data; modifying the metadata to reflect state changes related to a pending data transaction between the data storage space and the cache; writing the modified metadata to an atomic write descriptor; and recalling the atomic write descriptor for transacting a selected atomic block size in an event of an interruption while executing the data transaction.

Abstract: A data storage system adapted to maintain redundant data storage sets at a destination location(s) is disclosed. The data storage system establishes a copy set comprising a source volume and a destination volume. Data written to a source volume is automatically copied to the destination volume. The data storage system maintains a data log that may be activated when the destination volume is inaccessible due to, for example, a malfunction in the destination storage system or in the communication link between the source system and the destination system. I/O commands and the data associated with those commands are written to the data log, and after a destination system becomes available the information in the data log is merged into the destination volume to conform the data in the destination volume to the data in the source volume. The data log competes for disk capacity with other volumes on the system, and log memory is allocated as needed.

Abstract: A storage controller uses a stripe-aligned metadata to describe data buffers in the storage controller's cache. A stripe buffer list and other metadata structures are aligned to stripe boundaries on storage media. Each stripe buffer list references data buffers in the cache memory of the storage controller that are dedicated to storing the data associated with a given disk stripe during a storage transaction. A stripe buffer descriptor table associates each stripe buffer list with the corresponding disk stripe through a virtual block address range and logical disk identifier. A stripe buffer list “present bits” table and a stripe buffer list “dirty bits” table maintain a record of the state of individual data buffers. With these metadata structures, data buffers in cache memory are effectively aligned with virtual disk stripes of the storage media, making for a simpler and more efficient storage transaction.

Abstract: Methods and systems for managing disk capacity allocated to a data log in a source data storage system during a merge process are disclosed. Data in the data log may be merged into corresponding data on a destination storage system connected to the source data storage system by at least one communication link. In one embodiment a method comprises maintaining a ratio of merge writes out of the data log to writes from a host computer into the data log within a desired range until the write process reaches a predetermined distance from the end of the data log, and quiescing writes from a host computer into the data log until the data log is fully written to the destination storage system.

Abstract: A data storage system is provided comprising memory allocation information associated with a change in a system configuration of memory allocation within a memory space, and means for updating the system configuration by saving the memory allocation information before updating the system configuration. A recovery record comprises memory allocation information associated with the change in system configuration of memory allocation, and a completion indicator comprises a first value if the memory allocation information is included in the system configuration and comprises a second value if the memory allocation information is not included in the system configuration.

Abstract: Metadata architecture and associated methodology for a data storage system employing a grid-based storage capacity wherein each grid defines a storage unit in terms of a plurality of storage domains along one axis against one or more rows of a plurality of storage stripes along another axis, and wherein a grid grouping of two or more grids defines a sheet of data storage capacity. The metadata comprises identification information stored in a memory space characterizing an allocation status of the sheets in the storage capacity. A method is provided for allocating memory for the grid-based storage capacity comprising determining whether the allocation is associated with an existing logical device. If the determining step is associated with an existing logical device, then a logical device allocation map and a sheet allocation table are accessed to allocate a selected storage unit. Otherwise, a sheet allocation descriptor and a sheet allocation map are accessed to allocate a selected storage unit.

Abstract: A generic storage container system is provided for a grid-based storage architecture, comprising a generic storage container comprising a plurality of storage domains along one axis against a plurality of rows of stripes along another axis defining a preselected storage capacity, and configuration information allocating the stripes in response to a storage format specified by an allocation request. A method is provided for storing the data, comprising: providing the generic storage container; providing configuration information adapted for selectively allocating the stripes in relation to a data storage format; specifying a desired storage format; and allocating the stripes in response to the desired format.

Abstract: A data storage system and associated method comprising system configuration information; a first processor adapted for identifying a portion of the system configuration information in response to a configuration change request to the memory space, and for signaling an update request incident with the configuration change request to a second processor; and a second processor adapted for updating the portion in response to the update request and independently of the first processor.

Abstract: Methods and systems for managing disk capacity allocated to a data log in a source data storage system during a merge process are disclosed. Data in the data log may be merged into corresponding data on a destination storage system connected to the source data storage system by at least one communication link. In one embodiment a method comprises maintaining a ratio of merge writes out of the data log to writes from a host computer into the data log within a desired range until the write process reaches a predetermined distance from the end of the data log, and quiescing writes from a host computer into the data log until the data log is fully written to the destination storage system.

Abstract: A data storage system adapted to maintain redundant data storage sets at a destination location(s) is disclosed. The data storage system establishes a copy set comprising a source volume and a destination volume. Data written to a source volume is automatically copied to the destination volume. The data storage system maintains a data log that may be activated when the destination volume is inaccessible due to, for example, a malfunction in the destination storage system or in the communication link between the source system and the destination system. I/O commands and the data associated with those commands are written to the data log, and after a destination system becomes available the information in the data log is merged into the destination volume to conform the data in the destination volume to the data in the source volume. The data log competes for disk capacity with other volumes on the system, and log memory is allocated as needed.

Abstract: A system and method for replicating data between a plurality of storage locations. Each storage location has a controller with a port to a data communication network and a pool of storage. A logical unit (LUN) of storage is implemented at each storage location such that each implemented LUN comprises a substantially complete replica of the data stored in each other LUN. At least two of the controllers at the plurality of storage locations present their implemented LUN as an active device to the one or more host systems for receiving operational data access requests. Preferably, there are three or more storage locations in the system. Preferably, there are three or more storage locations in the system and each storage location can behave as a source location or a destination location for a particular operational data transaction.

Abstract: A data storage system adapted to maintain redundant data storage sets at a destination location(s) is disclosed. The data storage system establishes a copy set comprising a source volume and a destination volume. Data written to a source volume is automatically copied to the destination volume. The data storage system maintains a data log that may be activated when the destination volume is inaccessible due to, for example, a malfunction in the destination storage system or in the communication link between the source system and the destination system. I/O commands and the data associated with those commands are written to the data log, and after a destination system becomes available the information in the data log is merged into the destination volume to conform the data in the destination volume to the data in the source volume. The data log competes for disk capacity with other volumes on the system, and log memory is allocated as needed.

Abstract: A data replication management (DRM) architecture comprising a plurality of storage cells interconnected by a fabric. Flexibility in connectivity is provided by configuring each storage cell port to the fabric to handle both host data access requests and DRM traffic. Each storage cell comprises one or more storage controllers that can be connected to the fabric in any combination. Processes executing in the storage controller find a path to a desired destination storage cell. The discovery algorithm implements a link service that exchanges information related to DRM between the storage controllers. The DRM architecture is symmetric and peer cooperative such that each controller and storage cell can function as a source and a destination of replicated data. The DRM architecture supports parallel and serial “fan-out” to multiple destinations, whereby the multiple storage cells may implement data replicas.