Abstract: In one embodiment, an extent hashing technique is used to efficiently distribute data and associated metadata substantially evenly among nodes of a cluster. The data may be write data associated with a write request issued by a host and received at a node of the cluster. The write data may be organized into one or more extents. A hash function may be applied to the extent to generate a result which may be truncated or trimmed to generate a hash value. A hash space of the hash value may be divided into a plurality of buckets representative of the write data, i.e., the extents, and the associated metadata, i.e., extent metadata. A number of buckets may be assigned to each extent store instance of the nodes to distribute ownership of the buckets, along with their extents and extent metadata, across all of the extent store instances of the nodes.

Abstract: A file system layout apportions an underlying physical volume into one or more virtual volumes (vvols) of a storage system. The underlying physical volume is an aggregate comprising one or more groups of disks, such as RAID groups, of the storage system. The aggregate has its own physical volume block number (pvbn) space and maintains metadata, such as block allocation structures, within that pvbn space. Each vvol has its own virtual volume block number (vvbn) space and maintains metadata, such as block allocation structures, within that vvbn space. Notably, the block allocation structures of a vvol are sized to the vvol, and not to the underlying aggregate, to thereby allow operations that manage data served by the storage system (e.g., snapshot operations) to efficiently work over the vvols. The file system layout extends the file system layout of a conventional write anywhere file layout system implementation, yet maintains performance properties of the conventional implementation.

Abstract: In one embodiment, a node coupled to one or more storage devices executes a storage input/output (I/O) stack having a volume layer. The volume layer manages volume metadata embodied as mappings from offsets of a logical unit (LUN) to extent keys associated with storage locations for extents on the one or more storage devices. Volume metadata is maintained as a dense tree metadata structure representing successive points in time. The dense tree metadata structure has multiple levels, wherein a top level of the dense tree metadata structure represents newer volume metadata changes and descending levels of the dense tree metadata structure represent older volume metadata changes. The node accesses a latest version of changes to the volume metadata by searching from the top level to the descending levels in the dense tree metadata structure.

Abstract: The embodiments described herein are directed to an organization of metadata managed by a volume layer of a storage input/output (I/O) stack executing on one or more nodes of a cluster. The metadata managed by the volume layer, i.e., the volume metadata, is illustratively embodied as mappings from addresses, i.e., logical block addresses (LBAs), of a logical unit (LUN) accessible by a host to durable extent keys maintained by an extent store layer of the storage I/O stack. In an embodiment, the volume layer organizes the volume metadata as a mapping data structure, i.e., a dense tree metadata structure, which represents successive points in time to enable efficient access to the metadata.

Abstract: In one embodiment, a flash-optimized, log-structured layer of a file system of a storage input/output (I/O) stack executes on one or more nodes of a cluster. The log-structured layer of the file system provides sequential storage of data and metadata on solid state drives (SSDs) to reduce write amplification, while leveraging variable compression and variable length data features of the storage I/O stack. The data may be organized as an arbitrary number of variable-length extents of one or more host-visible logical units (LUNs). The metadata may include mappings from host-visible logical block address ranges of a LUN to extent keys, as well as mappings of the extent keys to SSD storage locations of the extents. The storage location of an extent on SSD is effectively “virtualized” by its mapped extent key such that relocation of the extent on SSD does not require update to volume layer metadata.

Abstract: A method for sharing data blocks in a hierarchical file system in a storage server includes allocating a plurality of data blocks in the file system, and sharing data blocks in the file system, without using a persistent point-in-time image, to avoid duplication of data blocks. A method for identifying data blocks that can be shared includes computing a fingerprint for each of multiple data blocks to be written to a storage facility and storing the fingerprint with information identifying the data block in an entry in a set of metadata. The set of metadata is used to identify data blocks which are duplicates.

Abstract: A file system layout apportions an underlying physical volume into one or more virtual volumes (vvols) of a storage system. The underlying physical volume is an aggregate comprising one or more groups of disks, such as RAID groups, of the storage system. The aggregate has its own physical volume block number (pvbn) space and maintains metadata, such as block allocation structures, within that pvbn space. Each vvol has its own virtual volume block number (vvbn) space and maintains metadata, such as block allocation structures, within that vvbn space. Notably, the block allocation structures of a vvol are sized to the vvol, and not to the underlying aggregate, to thereby allow operations that manage data served by the storage system (e.g., snapshot operations) to efficiently work over the vvols. The file system layout extends the file system layout of a conventional write anywhere file layout system implementation, yet maintains performance properties of the conventional implementation.

Abstract: A file system layout apportions an underlying physical volume into one or more virtual volumes (vvols) of a storage system. The underlying physical volume is an aggregate comprising one or more groups of disks, such as RAID groups, of the storage system. The aggregate has its own physical volume block number (pvbn) space and maintains metadata, such as block allocation structures, within that pvbn space. Each vvol has its own virtual volume block number (vvbn) space and maintains metadata, such as block allocation structures, within that vvbn space. Notably, the block allocation structures of a vvol are sized to the vvol, and not to the underlying aggregate, to thereby allow operations that manage data served by the storage system (e.g., snapshot operations) to efficiently work over the vvols.

Abstract: A method for sharing data blocks in a hierarchical file system in a storage server includes allocating a plurality of data blocks in the file system, and sharing data blocks in the file system, without using a persistent point-in-time image, to avoid duplication of data blocks. A method for identifying data blocks that can be shared includes computing a fingerprint for each of multiple data blocks to be written to a storage facility and storing the fingerprint with information identifying the data block in an entry in a set of metadata. The set of metadata is used to identify data blocks which are duplicates.

Abstract: A file system layout apportions an underlying physical volume into one or more virtual volumes (vvols) of a storage system. The underlying physical volume is an aggregate comprising one or more groups of disks, such as RAID groups, of the storage system. The aggregate has its own physical volume block number (pvbn) space and maintains metadata, such as block allocation structures, within that pvbn space. Each vvol has its own virtual volume block number (vvbn) space and maintains metadata, such as block allocation structures, within that vvbn space. Notably, the block allocation structures of a vvol are sized to the vvol, and not to the underlying aggregate, to thereby allow operations that manage data served by the storage system (e.g., snapshot operations) to efficiently work over the vvols.

Abstract: An underlying physical volume of a storage system is an aggregate having a plurality of storage devices. The aggregate has its own physical volume block number (pvbn) space. A file system layout apportions the underlying physical volume into a plurality of virtual volumes of the storage system each having a virtual volume identification (vvid). Each virtual volume has its own virtual volume block number (vvbn) space. The block allocation structures of a virtual volume are sized to the virtual volume, and not to the underlying aggregate, to thereby allow operations that manage data served by the storage system (e.g., snapshot operations) to efficiently work over the virtual volumes. Each storage block in a virtual volume is identified by the triplet: pvbn, vvid, and vvbn.

Abstract: A method and apparatus for operating a data storage system is disclosed. An original active file system holds incoming write transactions. Data is written at a selected time to blocks in a data storage device of the original active file system, the data written to blocks which do not hold old data of the data storage system. Pointers to data of the original active file system are written at the selected time to the data storage device, the pointers written to blocks which do not hold old data of the data storage system, the pointers and a previously saved data of the active file system forming a consistency point of the original active file system at the selected time. A new active file system is started using the consistency point of the original active file system at the selected time.

Abstract: A system and method for operating a storage system is provided. A plurality of operating system transaction entries are stored in a log, and a swarm of messages with respect to the plurality of operating system transaction entries is established. The swarm of messages is delivered to an operating system of the storage system. A processor performs a parallel retrieval process for a plurality of messages in the swarm of messages by processing the plurality of messages in an arbitrary order without regard to an underlying order of the messages.

Abstract: The present invention provides a technique for efficiently generating and validating external file handles at a storage operating system of a filer. According to one aspect of the technique, the storage operating system associates a unique internal file handle with each file or directory stored in the filer. This unique internal file handle is used to generate a corresponding external file handle. Unlike previous approaches, the internal file handles' representations do not change depending on the location of their corresponding files or directories, e.g., on-disk or in-core, within the filer, thereby simplifying file management within the filer. According to another aspect of the inventive technique, the filer executes a file-handle validation procedure to determine whether an external file handle received with a file access request has become invalid (or “stale”) as a result of an NVRAM failure in the filer.

Abstract: A system and method provides continuous data protection using checkpoints in a write anywhere file system. During a consistency point of a write anywhere file system, freed blocks are identified and are appended to a delete log for retention. A consistency point log is updated with a new entry associated with the consistency point. If the file system needs to retrieve its state at a particular point in time, the stored blocks of the delete log may be recovered.

Abstract: A system and method for enabling parallel replay of a backup memory log of client transaction request entries to a network storage appliance file system is provided. The backup memory is typically implemented as a non-volatile random access memory (NVRAM). An initiator establishes a swarm of messages with a plurality of transaction blocks pointing to logged request entries and related states associated therewith. The states represent the various phases of file system recovery and disk storage including a retrieval of disk information (data and meta-data), typically in the form of a LOAD, and a subsequent modify phase. The swarm is transferred to the file system for parallel disk information-retrieval in an interleaved process. Any transactions that cannot be performed due to a required prerequisite action (e.g. a prior file-create) are returned to the initiator for reloading once the prerequisite action has occurred.

Abstract: A system and method for operating a storage system is provided. A plurality of operating system transaction entries are stored in a log, and a swarm of messages with respect to the plurality of operating system transaction entries is established. The swarm of messages is delivered to an operating system of the storage system. A processor performs a parallel retrieval process for a plurality of messages in the swarm of messages by processing the plurality of messages in an arbitrary order without regard to an underlying order of the messages.

Abstract: A system and method automatically generates a root volume for use by a storage operating system of a storage system. During initialization of the storage system, the storage operating system detects that no operational root volume exists and, in response, invokes a root volume generation process. The root volume generation process automatically creates a thinly provisioned flexible volume of the smallest allowable size on an aggregate. The created flexible volume is marked as the root volume for the storage system. The storage operating system then continues initialization utilizing the automatically generated root volume.

Abstract: A file system layout apportions an underlying physical volume into one or more virtual volumes (vvols) of a storage system. The underlying physical volume is an aggregate comprising one or more groups of disks, such as RAID groups, of the storage system. The aggregate has its own physical volume block number (pvbn) space and maintains metadata, such as block allocation structures, within that pvbn space. Each vvol has its own virtual volume block number (vvbn) space and maintains metadata, such as block allocation structures, within that vvbn space. Notably, the block allocation structures of a vvol are sized to the vvol, and not to the underlying aggregate, to thereby allow operations that manage data served by the storage system (e.g., snapshot operations) to efficiently work over the vvols. The file system layout extends the file system layout of a conventional write anywhere file layout system implementation, yet maintains performance properties of the conventional implementation.

Abstract: A system and method for enabling parallel replay of a backup memory log of client transaction request entries to a network storage appliance file system is provided. The backup memory is typically implemented as a non-volatile random access memory (NVRAM). An initiator establishes a swarm of messages with a plurality of transaction blocks pointing to logged request entries and related states associated therewith. The states represent the various phases of file system recovery and disk storage including a retrieval of disk information (data and meta-data), typically in the form of a LOAD, and a subsequent modify phase. The swarm is transferred to the file system for parallel disk information-retrieval in an interleaved process. Any transactions that cannot be performed due to a required prerequisite action (e.g. a prior file-create) are returned to the initiator for reloading once the prerequisite action has occurred.