Abstract: The invention provides a method and system for routing or switching file system requests in a mass storage system. The mass storage system includes multiple storage devices coupled using fiber channel to a routing or switching device disposed internally within each shelf of mass storage devices in the mass storage system, and coupled directly to each individual mass storage device in that shelf. The switching device receives requests, identifies an individual target storage device for each such request, and sends each such request to its designated target storage device. The switching device also receives responses to such requests from each individual storage device within the shelf, and sends those responses from the storage device to an external medium with which the response can be delivered to the original requester.

Abstract: On disk failure, the storage system migrates only those disk blocks that included allocated data, and treats unallocated disk blocks as being logically zero when possible. When there is no spare disk, the source data block is logically set to zero and parity is recalculated for the RAID stripe associated with the source data block. When there is a spare, unallocated blocks on the spare are logically or physically set to zero upon migration. Write operations for the failed disk are redirected to other non-failing disks, and a record of which in-use disk blocks have been thus “migrated” to those other non-failing disks in maintained. Unused disk blocks are proactively set to zero. A target mirror copy is created using information regarding allocated disk blocks, by copying those blocks including allocated data or parity, and by clearing at the mirror those blocks not including any allocated data or parity.

Abstract: The invention provides a method and system for recovery of file system data in file servers having mirrored file system volumes. The invention makes use of a “snapshot” feature of a robust file system (the “WAFL File System”) disclosed in the Incorporated Disclosures, to rapidly determined which of two or more mirrored volumes is most up-to-date, and which file blocks of the most recent mirrored volume have been changed from each one of the mirrored file systems. In a preferred embodiment, among a plurality of mirrored volumes, the invention rapidly determines which is the most up-to-date by examining a consistency point number maintained by the WAFL File System at each mirrored volume.

Abstract: The invention provides techniques for duplicating all or part of a file system while maintaining consistent copies of the file system. A preferred embodiment reads data from the source file system (mapped according to a first storage block arrangement) creates an image stream and writes this data onto the destination file system (in accordance with a second storage block arrangement). Thus, the invention allows an image transfer between file systems that have dissimilar characteristics. An intermediate storage medium can be used to store or transmit the data read from the source file system before writing the data to the destination file system. (For example, the intermediate storage medium can include a magnetic tape or other archival device, or a network.) Another aspect of the invention writes the data to the intermediate storage medium using a normalized or implied block list.

Abstract: A system and method for remote asynchronous replication or mirroring of changes in a source file system snapshot in a destination replica file system using a scan (via a scanner) of the blocks that make up two versions of a snapshot of the source file system, which identifies changed blocks in the respective snapshot files based upon differences in volume block numbers identified in a scan of the logical file block index of each snapshot. Trees of blocks associated with the files are traversed, bypassing unchanged pointers between versions and walking down to identify the changes in the hierarchy of the tree. These changes are transmitted to the destination mirror or replicated snapshot. This technique allows regular files, directories, inodes and any other hierarchical structure to be efficiently scanned to determine differences between versions thereof.

Abstract: The invention features a method for controlling storage of data in a plurality of storage devices each including storage blocks, for example, in a RAID array. The method includes receiving a plurality of write requests associated with data, and buffering the write requests. A file system defines a group of storage blocks, responsive to disk topology information. The group includes a plurality of storage blocks in each of the plurality of storage devices. Each data block of the data to be written is associated with a respective one of the storage blocks, for transmitting the association to the plurality of storage devices.

Abstract: The invention provides a storage system, and a method for operating a storage system, that provides for relatively rapid and reliable takeover among a plurality of independent file servers. Each file server maintains a reliable communication path to the others. Each file server maintains its own state in reliable memory. Each file server regularly confirms the state of the other file servers. Each file server labels messages on the redundant communication paths, so as to allow other file servers to combine the redundant communication paths into a single ordered stream of messages. Each file server maintains its own state in its persistent memory and compares that state with the ordered stream of messages, so as to determine whether other file servers have progressed beyond the file server's own last known state.

Abstract: A technique efficiently corrects multiple storage device failures in a storage array using a combination of a single diagonal parity group and multiple row parity groups. The storage array comprises a plurality of concatenated sub-arrays, wherein each sub-array includes a set of data storage devices and a parity storage device. Each row parity group is associated with a sub-array of the array. The array further includes a global parity storage device holding diagonal parity computed across the concatenation of the sub-arrays. Instead of requiring that each parity group contain both a row parity device and a diagonal parity device, the array is composed of a collection of row parity groups. Diagonal parity is calculated across the full array.

Abstract: A “row-diagonal” (R-D) parity technique reduces overhead of computing diagonal parity for a storage array adapted to enable efficient recovery from the concurrent failure of two storage devices in the array. The diagonal parity is computed along diagonal parity sets that collectively span all data disks and a row parity disk of the array. The parity for all of the diagonal parity sets except one is stored on the diagonal parity disk. The R-D parity technique provides a uniform stripe depth and an optimal amount of parity information.

Abstract: The invention provides techniques for duplicating all or part of a file system while maintaining consistent copies of the file system. A preferred embodiment reads data from the source file system (mapped according to a first storage block arrangement) creates an image stream and writes this data onto the destination file system (in accordance with a second storage block arrangement). Thus, the invention allows an image transfer between file systems that have dissimilar characteristics. An intermediate storage medium can be used to store or transmit the data read from the source file system before writing the data to the destination file system. (For example, the intermediate storage medium can include a magnetic tape or other archival device, or a network.) Another aspect of the invention writes the data to the intermediate storage medium using a normalized or implied block list.

Abstract: In one embodiment, a first storage device and a second storage device form a mirror. When the first storage device loses synchronization with the second storage device, data present in the second storage device but not in the first storage device are identified. The identified data are then copied to the first storage device. In one embodiment, a method of rebuilding data in a storage device includes the act of replacing a failed storage device with a replacement storage device. Up-to-date data for the failed storage device, which may be stored in a corresponding mirror, may then be copied to the replacement storage device. Thereafter, the replacement storage device and any other storage devices that have lost synchronization with their mirror are resynchronized.

Abstract: In one embodiment, a first storage device and a second storage device form a mirror. When the first storage device loses synchronization with the second storage device, data present in the second storage device but not in the first storage device are identified. The identified data are then copied to the first storage device.

Abstract: The invention provides a storage system, and a method for operating a storage system, that provides for relatively rapid and reliable takeover among a plurality of independent file servers. Each file server maintains a reliable communication path to the others. Each file server maintains its own state in reliable memory. Each file server regularly confirms the state of the other file servers. Each file server labels messages on the redundant communication paths, so as to allow other file servers to combine the redundant communication paths into a single ordered stream of messages. Each file server maintains its own state in its persistent memory and compares that state with the ordered stream of messages, so as to determine whether other file servers have progressed beyond the file server's own last known state.

Abstract: The invention provides a method and system for duplicating all or part of a file system while maintaining consistent copies of the file system. The file server maintains a set of snapshots, each indicating a set of storage blocks making up a consistent copy of the file system as it was at a known time. Each snapshot can be used for a purpose other than maintaining the coherency of the file system, such as duplicating or transferring a backup copy of the file system to a destination storage medium. In a preferred embodiment, the snapshots can be manipulated to identify sets of storage blocks in the file system for incremental backup or copying, or to provide a file system backup that is both complete and relatively inexpensive.

Abstract: The invention provides a storage system that is highly available even in the face of component failures in the storage system, and a method for operating that storage system. A first and a second file server each includes a file server request log for storing incoming file server requests. Both the first and second file servers have access to a common set of mass storage elements. Each incoming file server request is copied to both the first and second file servers; the first file server processes the file server request while the second file server maintains a copy in its file server request log. Each file server operates using a file system that maintains consistent state after each file server request. On failover, the second file server can perform those file server requests in its file server request log since the most recent consistent state. There is no single point of failure that prevents access to any individual mass storage element.

Abstract: A method and apparatus for bypassing multiple pre-tests and post-tests during a system call when those tests are known to be inapplicable. One or more slow path flags are checked during a system call or TRAP. If the slow path flag is clear, execution follows a fast instruction path, resulting in faster execution for the system call or TRAP. Otherwise execution follows a slow instruction path. The slow path flags are set, cleared, and checked at appropriate times. The invention improves the execution time of a thread in a software process and may be used in a data processing system employing multiple threads. Each thread in the data processing system has its own set of slow path flags. The invention can set, clear and check the slow path flags of each thread independently, in subsets of threads, or in all threads.

Abstract: Doors are implemented in an existing operating system by using a novel adaptation of existing data structures and procedures which represent and control data files of the operating system. A door is represented by a file descriptor and is managed by existing mechanisms of an operating system, such as the Unix operating system, which are also used to manage files. A door is represented by a door node, which is a novel extension of the conventional vnode structure which in turn is used in conventional operating systems to represent the state of a computer resource, e.g., a data file. A door node is adapted such that most conventional file access and control procedures, e.g., procedures open( ) and close( ), can be used to access and control door with minimal adaptation.

Abstract: The invention provides a storage system, and a method for operating a storage system, that provides for relatively rapid and reliable takeover among a plurality of independent file servers. Each file server maintains a reliable communication path to the others. Each file server maintains its own state in reliable memory. Each file server regularly confirms the state of the other file servers. Each file server labels messages on the redundant communication paths, so as to allow other file servers to combine the redundant communication paths into a single ordered stream of messages. Each file server maintains its own state in its persistent memory and compares that state with the ordered stream of messages, so as to determine whether other file servers have progressed beyond the file server's own last known state.

Abstract: The invention provides a method and system for performing XOR operations without consuming substantial computing resources. A specialized processor is coupled to the same bus as a set of disk drives; the specialized processor reviews data transfers to and from the disk drives and performs XOR operations on data transferred to and from the disk drives without requiring separate transfers. The specialized processor maintains an XOR accumulator which is used for XOR operations, which records the result of XOR operations, and which is read out upon command of the processor. The XOR accumulator includes one set of accumulator registers for each RAID stripe, for a selected set of RAID stripes. A memory (such as a contents-addressable memory) associates one set of accumulator registers with each selected RAID stripe.

Abstract: The present invention is a system and method for "checking the use of synchronization locks in a multi-threaded computer program" (hereinafter "WARLOCK II"). In Warlock II a set of source code representing a process which includes multiple threads may be annotated by the developer using a "NOTE" macro to describe the desired operation of the locks. This annotated source code is passed to a modified ANSI C compiler which outputs a special file designated a ".ll" file, as for example, file "foo.ll". This special ".ll" file is then processed as input along with other designated ".ll" files which might be related, by a "wlanalyze" program which will check the source code of the target programs in the ".ll" files to determine whether their use of synchronization locks is consistent with the intended use as specified in the annotations.