Abstract: The present invention provides a system and a method for utilizing a parity protection module to back up data on striped aggregates. Specifically, the system computes parity data for data stored at a particular location of each of a plurality of constituent aggregates, and stores the parity on one of the constituent aggregates that is a parity owner for that particular location of data. In the event one of the constituent aggregates fails, new data may still be accessed by the system (the striped aggregates), both to write new data, and to read data stored on the failed aggregate. In particular, the parity protection module allows clients to read data from a failed aggregate by running a reverse parity computation, which may also be used to restore the data to the failed aggregate.

Abstract: A system and method addresses data containers in a clustered storage system. Each instantiation of a volume, or other container set, is associated with a data set identifier (DSID). All instantiations of data that represent a single point in time are associated with a master data set identifier (MSID). A volume location database (VLDB), utilizing a replicated database among the nodes of the cluster, stores a data set data structure containing appropriate mapping information between the MSIDs and DSIDs. Clients of the clustered storage system utilize MSIDs to reference data that is desired to be accessed. When a storage system receives a data access request containing a MSID, the storage system accesses the VLDB to identify an appropriate DSID to which to forward the request. The data access request is then forwarded to the appropriate storage system for processing.

Abstract: A technique for accelerating the creation of a hyperclone of a virtual file system. Specifically, each data block and indirect block directly referenced by a base inode are only copied. The hyperclone may then reference the original data blocks and original indirect blocks. Further, an indication may be provided for at least one of the copied data blocks and indirect blocks that indicates that the at least one of the copied data blocks and indirect blocks is in use.

Abstract: The present invention provides a mechanism for storing and accessing attributes of a data container (e.g., characteristics such as a name of a data container used to perform a lookup operation against the data container) in cache memory within storage systems in a cluster of storage systems. Further, a mechanism is provided so that an attribute of a data container is inserted into a cache memory of preferably one storage system (in the cluster) that caches metadata of the data container (e.g., information describing the data container). As a result, a lookup operation for a data container is directed to and served by the storage system that caches the metadata and the attribute of the data container. The lookup request is not relayed to a storage system that maintains a central directory of attributes of all data containers in the cluster.

Abstract: The technique introduced here includes generating a data structure for use in determining how responsibilities for services should be distributed amongst a plurality of resources in a network storage system. The technique includes an iterative process of optimizing the data structure for a plurality of performance/quality metrics, such as evenness of storage consumption across the storage system or a designated subset thereof, number of “hot spots”, degree of data scatter, and number of changes needed to reflect a change in storage system geometry. The data structure can be a striping table for striping logical containers of data across multiple storage resources, such as data volumes, or physical storage devices. The “responsibilities for services”, can include responsibility for storing a stripe of a logical container of data or responsibility for storing a segment of parity data for a logical container of data.

Abstract: A data distribution technique is configured to provide capacity leveling in a striped file system. When a new node is added to a striped volume set, the striping table is evolved to accommodate the newly added node. Each node of a cluster is illustratively associated with a capacity value that takes into account, e.g., processor speed, number of processors, hardware configuration and/or software available for the node. During the evolution process of the striping table, the technique apportions stripes of the SVS among the nodes in a manner so that they are optimally assigned to the nodes in accordance with each node's capacity value. By utilizing the evolutionary striping table that incorporates capacity values, heterogeneous nodes may be utilized to their maximum capacity within a striped volume set, thereby reducing underutilized processing resources.

Abstract: A computer data storage system is described. A processor maintains a striped volume set by striping a data container over a plurality of storage nodes. A storage node determines whether space available on that node is below a predetermined threshold, the predetermined threshold indicating a low-in-space state. The storage node sends a message indicating that the storage node is in a low-in-space state. The processor accepts no further write messages to the data container as long as the storage node is in a low-in-space state.

Abstract: Apparatus and method are disclosed for managing lock state information in a distributed file system. A set of data containers storing data is maintained. The data containers are striped across a plurality of volumes, where each volume includes one or more data storage devices. A metadata volume is maintained with the plurality of volumes, the metadata volume configured to include a lock state database that stores lock state information for the plurality of volumes. The lock state information is communicated between the metadata volume and the plurality of volumes.

Abstract: Provided is a method and system for performing volume migrations while retaining persistent point in time data of the migrated volume. Specifically, by using a collective persistent point in time data map that describes a collection of persistent point in time data of a volume to be migrated, the collective persistent point in time data map and volume data can be migrated during a quiesced period of user activity. Then, during a non-quiesced period of user activity, the persistent point in time data can be transferred to the location of the migrated volume in real-time.

Abstract: A system and method revokes soft locks in a distributed system. A data volume of the distributed system determines that one or more soft locks need to be revoked in order for an operation to be processed. The data volume transmits a request to revoke the soft locks to a meta-data volume that stores a canonical copy of meta-data.

Abstract: The present invention provides a mechanism for storing and accessing attributes of a data container (e.g., characteristics such as a name of a data container used to perform a lookup operation against the data container) in cache memory within storage systems in a cluster of storage systems. Further, a mechanism is provided so that an attribute of a data container is inserted into a cache memory of preferably one storage system (in the cluster) that caches metadata of the data container (e.g., information describing the data container). As a result, a lookup operation for a data container is directed to and served by the storage system that caches the metadata and the attribute of the data container.

Abstract: A storage system architecture ensures transactional processing of operations directed to one or more data containers stored on a plurality of volumes distributed across a plurality of nodes interconnected as a cluster. The volumes are organized as a striped volume set (SVS) comprising one meta-data volume configured to store a canonical copy of certain meta-data associated with all data containers stored on the SVS, and one or more data volumes configured to store, at least, data content of those containers. In addition, for each data container stored on the SVS, one volume is designated a container attribute volume and, as such, is configured to store a canonical copy of certain, rapidly-changing attribute meta-data associated with that container. Notably, the invention provides a plurality of SVS operations that enables transactional performance in the cluster using persistent storage and/or systematic accesses to the data/meta-data stored on the SVS volumes.

Abstract: A system and method re-stripes one or more data containers across a striped volume set (SVS) that has been modified by the addition of one or more volumes. The SVS is associated with an existing set of striping rules that define a stripe algorithm, a stripe width and an ordered list of volumes distributed across a plurality of nodes interconnected as a cluster. Each node of the cluster includes (i) a disk element (D-blade) adapted to service a volume of the SVS and (ii) a network element (N-blade) adapted to redirect a data access request to any D-blade of the cluster. Notably, the content of each data container is apportioned among the volumes of the SVS to thereby improve the efficiency of storage service provided by the cluster. To that end, the stripe algorithm specifies the manner in which the data container content is apportioned as stripes across the plurality of volumes, while the stripe width specifies the size/width of each stripe.

Abstract: A data distribution technique is configured to provide capacity leveling in a striped file system. When a new node is added to a striped volume set, the striping table is evolved to accommodate the newly added node. Each node of a cluster is illustratively associated with a capacity value that takes into account, e.g., processor speed, number of processors, hardware configuration and/or software available for the node. During the evolution process of the striping table, the technique apportions stripes of the SVS among the nodes in a manner so that they are optimally assigned to the nodes in accordance with each node's capacity value. By utilizing the evolutionary striping table that incorporates capacity values, heterogeneous nodes may be utilized to their maximum capacity within a striped volume set, thereby reducing underutilized processing resources.

Abstract: A method and system for promoting a snapshot in a distributed striped volume system is provided. A master volume server is configured with a rollback process such that when it is determined that a rollback is required, the master volume server sets a flag persistently in its own raid label on disk. After the persistent flag is set, the master volume server determines a “common snapshot,” and starts the process of sending RPC messages to each node hosting constituent volumes instructing each constituent volume to roll back to the identified snapshot. When the nodes receive this message a flag is set in the own raid label of each constituent volume and the volume then promotes the particular snapshot. If the master volume server has not received a successful response from each node that the snapshot promotion was successful within a specified time period, there is a retry. The common snapshot is then used as the active file system, thus providing data recovery for the striped volume set.

Abstract: A system and method for managing space availability in a distributed striped file system is provided. A master data server is configured to send space availability detection messages to a plurality of data volumes servers hosting constituent volumes of a striped volume set. If one of the constituent volumes in the striped volume set has a low-in-space flag set, then the master data volume instructs all of the constituent volumes to set a low-in-space required flag, and no further writes are accepted for the striped volume set. The low-in-space and low-in-space required flags represent two states, and these states are returned in response to subsequent space availability detection messages from the master data server. A procedure for utilizing reserved space to complete an accepted cross stripe write operation is also provided.

Abstract: The present invention provides a system and a method for utilizing a parity protection module to back up data on striped aggregates. Specifically, the system computes party data for data stored at a particular location of each of a plurality of constituent aggregates, and stores the parity on one of the constituent aggregates that is a parity owner for that particular location of data. In the event one of the constituent aggregates fails, new data may still be accessed by the system (the striped aggregates), both to write new data, and to read data stored on the failed aggregate. In particular, the parity protection module allows clients to read data from a failed aggregate by running a reverse parity computation, which may also be used to restore the data to the failed aggregate.

Abstract: A system and method enables de-duplication in a storage system architecture comprising one or more volumes distributed across a plurality of nodes interconnected as a cluster. De-duplication is enabled through the use of file offset indexing in combination with data content redirection. File offset indexing is illustratively embodied as a Locate by offset function, while data content redirection is embodied as a novel Locate by content function. In response to input of, inter alia, a data container (file) offset, the Locate by offset function returns a data container (file) index that is used to determine a storage server that is responsible for a particular region of the file. The Locate by content function is then invoked to determine the storage server that actually stores the requested data on disk. Notably, the content function ensures that data is stored on a volume of a storage server based on the content of that data rather than based on its offset within a file.

Abstract: A system and method performs cross stripe write operations to a striped volume set (SVS) comprising of a plurality of volumes. A cross stripe write operation is directed to a first volume striping module (VSM) serving a first data volume where the entire write data of the operation is written to the volume. The first VSM then forwards excess data that extends beyond the boundary of a stripe to a second VSM for storage on a second data volume.

Abstract: A system and method caches and distributes meta-data for one or more data containers stored on a plurality of volumes configured as a striped volume set (SVS) and served by a plurality of nodes interconnected as a cluster. The SVS comprises one meta-data volume (MDV) configured to store a canonical copy of certain meta-data, including access control lists and directories, associated with all data containers stored on the SVS, and one or more data volumes (DV) configured to store, at least, data content of those containers. In addition, for each data container stored on the SVS, one volume is designated a container attribute volume (CAV) and, as such, is configured to store (“cache”) a canonical copy of certain, rapidly-changing attribute meta-data, including time stamps and container length, associated with that container.