Abstract: A system and method addresses data containers in a clustered storage system. Each instantiation of a volume, or other container set, is associated with a data set identifier (DSID). All instantiations of data that represent a single point in time are associated with a master data set identifier (MSID). A volume location database (VLDB), utilizing a replicated database among the nodes of the cluster, stores a data set data structure containing appropriate mapping information between the MSIDs and DSIDs. Clients of the clustered storage system utilize MSIDs to reference data that is desired to be accessed. When a storage system receives a data access request containing a MSID, the storage system accesses the VLDB to identify an appropriate DSID to which to forward the request. The data access request is then forwarded to the appropriate storage system for processing.

Abstract: A method for constructing an array of storage devices is disclosed. A number of storage devices is selected. Each storage device stores data in data blocks. A redundancy is established, the redundancy being a number of data blocks for a parity block. A plurality of parity sets is established, a parity set having the number of data blocks for a parity block and having a set parity block. The set parity block for each parity set is computed. Each set parity block is stored on one of the data storage devices that does not store one of the data-block members of the parity set, to store more than one parity block on each storage device, with an exception that two parity blocks for the same data storage location are precluded from being stored on the same data storage device.

Abstract: A technique provides an administrative path for accessing a writeable master storage volume in a mirrored storage environment. Illustratively, a writeable master storage volume stores a master set of data addressable by a corresponding pathname, and zero or more read-only (e.g., load-balancing) mirrored storage volumes are configured to store a mirrored set of the master set of data, the mirrored set also addressable by the corresponding pathname. Clients may read data from either the master storage volume or one of the mirrored storage volumes (e.g., according to a configured access location, such as a local mirrored volume if one exists) by issuing read requests having the corresponding pathnames. Also, each client may specifically access the master set of data from the master storage volume by issuing an access (e.g., read/write) request having a specified master storage volume pathname prefix prepended to the corresponding pathname.

Abstract: An n-way parity protection technique enables recovery of up to n storage device (e.g., disk) failures in a parity group of a storage array encoded to protect against n-way disk failures. The storage array is created by first configuring the array with m data disks, where m=p?1 and p is a prime number and a row parity disk. n?1 diagonal parity disks are then added to the array. Each diagonal parity set (i.e., diagonal) is associated with a slope that defines the data and row parity blocks of the array that are included in the diagonal. All diagonals having a common slope within a parity group are organized as a diagonal parity class. For each diagonal parity class, a diagonal parity storage disk is provided to store the diagonal parity.

Abstract: A triple parity (TP) technique reduces overhead of computing diagonal and anti-diagonal parity for a storage array adapted to enable efficient recovery from the concurrent failure of three storage devices in the array. The diagonal parity is computed along diagonal parity sets that collectively span all data disks and a row parity disk of the array. The parity for all of the diagonal parity sets except one is stored on the diagonal parity disk. Similarly, the anti-diagonal parity is computed along anti-diagonal parity sets that collectively span all data disks and a row parity disk of the array. The parity for all of the anti-diagonal parity sets except one is stored on the anti-diagonal parity disk. The TP technique provides a uniform stripe depth and an optimal amount of parity information.

Abstract: A system and method to allow a volume to appear in multiple locations in a super-namespace. A super-namespace is a hierarchy of namespaces within a storage system environment, such as a storage system cluster. The volume can appear in multiple locations in the super-namespace and/or in multiple namespaces within the super-namespace.

Abstract: A file system layout apportions an underlying physical volume into one or more virtual volumes (vvols) of a storage system. The underlying physical volume is an aggregate comprising one or more groups of disks, such as RAID groups, of the storage system. The aggregate has its own physical volume block number (pvbn) space and maintains metadata, such as block allocation structures, within that pvbn space. Each vvol has its own virtual volume block number (vvbn) space and maintains metadata, such as block allocation structures, within that vvbn space. Notably, the block allocation structures of a vvol are sized to the vvol, and not to the underlying aggregate, to thereby allow operations that manage data served by the storage system (e.g., snapshot operations) to efficiently work over the vvols.

Abstract: A triple parity (TP) technique reduces overhead of computing diagonal and anti-diagonal parity for a storage array adapted to enable efficient recovery from the concurrent failure of three storage devices in the array. The diagonal parity is computed along diagonal parity sets that collectively span all data disks and a row parity disk of the array. The parity for all of the diagonal parity sets except one is stored on the diagonal parity disk. Similarly, the anti-diagonal parity is computed along anti-diagonal parity sets that collectively span all data disks and a row parity disk of the array. The parity for all of the anti-diagonal parity sets except one is stored on the anti-diagonal parity disk. The TP technique provides a uniform stripe depth and an optimal amount of parity information.

Abstract: A system and method integrates a persistent reply cache with operation of a file system executing on a storage system. In an illustrative embodiment, the persistent reply cache is embodied as a file comprising a combination of contents of (i) one or more enhanced non-volatile log records of non-volatile electronic storage, (ii) “dirty” in-core buffer cache data structures, and (iii) on-disk data structures corresponding to those dirty buffer cache data structures. Integration is achieved through atomic operation of the reply cache with the file system to store information associated with a reply, e.g., to a client request, on electronic storage implemented as in-core buffer cache memory of the storage system, as well as on magnetic storage implemented as disks of the system.

Abstract: A cluster storage system comprises a plurality of nodes that access a shared storage, each node having two or more failover partner nodes. A primary node produces write logs for received write requests and produces parity data for the write logs (storing the parity data to local non-volatile storage). By storing parity data rather than actual write logs, the non-volatile storage space within the cluster for storing write logs is reduced. Prior to failure of the primary node, the primary node also sub-divides the write logs into two or more sub-sets and distributes the sub-sets to the two or more partner nodes for storage at non-volatile storage devices. Thus, if the primary node fails, its write logs are already distributed among the partner nodes so each partner node may perform the allotted write logs on the storage, thus improving the response time to the primary node failure.

Abstract: A technique efficiently transmits credentials between network elements and disk elements in a clustered storage system. According to the novel technique, in response to a user request to access data served by a data element, a network element inserts (adds) a credential associated with the user to a network element credential cache and creates a corresponding credential handle that indexes the credential in that cache. The network element relays the credential and credential handle to the disk element, which adds the credential to a corresponding disk element credential cache at a location indexed by the corresponding credential handle. Requests may then be sent between the elements using the credential handle. In addition, the network element may further send a series of chained requests to the disk element for the same credential/credential handle with an indication that the requests are for the same credential without sending the credential or credential handle.

Abstract: An underlying physical volume of a storage system is an aggregate having a plurality of storage devices. The aggregate has its own physical volume block number (pvbn) space. A file system layout apportions the underlying physical volume into a plurality of virtual volumes of the storage system each having a virtual volume identification (vvid). Each virtual volume has its own virtual volume block number (vvbn) space. The block allocation structures of a virtual volume are sized to the virtual volume, and not to the underlying aggregate, to thereby allow operations that manage data served by the storage system (e.g., snapshot operations) to efficiently work over the virtual volumes. Each storage block in a virtual volume is identified by the triplet: pvbn, vvid, and vvbn.

Abstract: A method and apparatus for storing data on a computer data storage system are described. Two or more computers are coupled together to form a cluster of computers. One or more data storage devices are coupled to the two or more computers. One or more volumes of data storage devices are implemented on a plurality of computers of the two or more computers, where each volume is a logical arrangement of the one or more storage devices coupled to a selected computer of the plurality of computers. A data container holds data stored by the data storage system. The data container is striped over a plurality of the one or more volumes of data storage devices, whereby data stored in the data container is striped over a plurality of the volumes of data storage devices.

Abstract: A triple parity (TP) technique reduces overhead of computing diagonal and anti-diagonal parity for a storage array adapted to enable efficient recovery from the concurrent failure of three storage devices in the array. The diagonal parity is computed along diagonal parity sets that collectively span all data disks and a row parity disk of the array. The parity for all of the diagonal parity sets except one is stored on the diagonal parity disk. Similarly, the anti-diagonal parity is computed along anti-diagonal parity sets that collectively span all data disks and a row parity disk of the array. The parity for all of the anti-diagonal parity sets except one is stored on the anti-diagonal parity disk. The TP technique provides a uniform stripe depth and an optimal amount of parity information.

Abstract: A method for recovering from three failed data storage devices is disclosed. A plurality of data storage devices hold data, and a row parity storage device holds row parity for them. The data storage devices and the row parity storage device form a set of storage devices. A diagonal parity storage device and an anti-diagonal parity storage device hold parity computed diagonally over the set of storage devices. In the event of a failure of three data storage devices of the set of storage devices, a first failed storage device is chosen for first restoration. A missing block of the first failed storage device is computed by using the remaining set of storage devices, and the diagonal parity storage device, and the anti-diagonal parity storage device. The remaining two failed storage devices are restored by a diagonal parity restoration technique.

Abstract: A method for constructing an array of storage devices is disclosed. A number of storage devices is selected. Each storage device stores data in data blocks. A redundancy is established, the redundancy being a number of data blocks for a parity block. A plurality of parity sets is established, a parity set having the number of data blocks for a parity block and having a set parity block. The set parity block for each parity set is computed. Each set parity block is stored on one of the data storage devices that does not store one of the data-block members of the parity set, to store more than one parity block on each storage device, with an exception that two parity blocks for the same data storage location are precluded from being stored on the same data storage device.

Abstract: A storage system architecture ensures transactional processing of operations directed to one or more data containers stored on a plurality of volumes distributed across a plurality of nodes interconnected as a cluster. The volumes are organized as a striped volume set (SVS) comprising one meta-data volume configured to store a canonical copy of certain meta-data associated with all data containers stored on the SVS, and one or more data volumes configured to store, at least, data content of those containers. In addition, for each data container stored on the SVS, one volume is designated a container attribute volume and, as such, is configured to store a canonical copy of certain, rapidly-changing attribute meta-data associated with that container. Notably, the invention provides a plurality of SVS operations that enables transactional performance in the cluster using persistent storage and/or systematic accesses to the data/meta-data stored on the SVS volumes.

Abstract: A lightweight coherency control protocol ensures consistency of data containers, such as a file, and associated data buffers stored on one or more volumes served by a plurality of nodes, e.g., storage systems, connected as a cluster. Each data buffer is associated with a current cache sequence number comprising a load generation value and an update count value. The load generation value is incremented every time an inode of a file is loaded into a memory of the storage system. Once the inode is loaded and its load generation value is set, then the appropriate buffer of a buffer tree for the file is loaded into the memory. The update count value is incremented each time the buffer is updated with a write request/operation. Therefore, each buffer loaded into memory is tagged with the load generation value and an update count from the time that buffer is loaded.

Abstract: A dynamic parity distribution system and technique distributes parity across disks of an array. The dynamic parity distribution system includes a storage operating system that integrates a file system with a RAID system. In response to a request to store (write) data on the array, the file system determines which disks contain free blocks in a next allocated stripe of the array. There may be multiple blocks within the stripe that do not contain file system data (i.e., unallocated data blocks) and that could potentially store parity. One or more of those unallocated data blocks can be assigned to store parity, arbitrarily. According to the dynamic parity distribution technique, the file system determines which blocks hold parity each time there is a write request to the stripe. The technique alternately allows the RAID system to assign a block to contain parity when each stripe is written.

Abstract: An n-way parity protection technique enables recovery of up to n storage device (e.g., disk) failures in a parity group of a storage array encoded to protect against n-way disk failures. The storage array is created by first configuring the array with m data disks, where m=p?1 and p is a prime number and a row parity disk. n?1 diagonal parity disks are then added to the array. Each diagonal parity set (i.e., diagonal) is associated with a slope that defines the data and row parity blocks of the array that are included in the diagonal. All diagonals having a common slope within a parity group are organized as a diagonal parity class. For each diagonal parity class, a diagonal parity storage disk is provided to store the diagonal parity.