Abstract: An improved technique involves identifying the location of backed free blocks, i.e., blocks within an upper deck file system that have been provisioned from storage devices of a data storage system to a lower deck file system, allocated from the lower deck file system to the upper deck file system, and later freed from the upper deck file system. A storage processor accesses a set of data structures that identifies backed free blocks as opposed to free blocks that have not been written into and thus do not correspond to any allocated blocks in storage. Once the storage processor identifies the backed free blocks, the storage processor frees each block in the lower deck file system from which the respective backed free block was allocated. The storage processor then updates the set of data structures to indicate that the respective backed free block is now simply a free block.

Abstract: Directory attributes are provided so that the time for creating an incremental backup of a file system by a scan of the file system tree is proportional generally to the number of files that change between backups instead of the number of files in the file system. A tree modification attribute indicates whether or not any file in a directory tree has changed since the last backup. If no file has changed in the directory tree, then the entire tree is skipped during the scan for changed files. In addition, a list is compiled of the files in the directory that represent branches having at least one changed file, so that the list is scanned instead of the directory entries. When a file is changed for the first time since the last backup, the file is queued so that the directory attributes are updated in a background process.

Abstract: A technique for managing storage tiering in a data storage apparatus operates at a fine level of granularity in response to write requests as blocks are allocated for writing. As write requests arrive or are otherwise processed by the data storage apparatus, the data storage apparatus identifies a quality of service to be provided for satisfying each write request and allocates a set of blocks from storage devices of storage tiers that provide the identified quality of service. The data storage apparatus may then store the information specified in the write request in the newly allocated blocks.

Abstract: A technique for managing a file system includes dividing the addressable space of a file system into multiple subspaces: a data subspace for user data; an inode subspace for inodes describing the user data; and an MDB (metadata block) subspace for indirect blocks and/or directory structures pertaining to the user data. In response to storage requests to write user files to the file system, the file system stores user data of the files in the data subspace, stores inodes pertaining to the user files in the inode subspace, and stores indirect blocks and/or directories in the MDB subspace.

Abstract: An improved technique generates formatted file system metadata in increments only when metadata for those increments are specifically requested. A location on a storage volume designated for storing file system metadata is read and tested to determine whether it stores a predetermined pattern indicating an unformatted state. If the pattern is found, metadata for the designated location are generated and returned. Otherwise, metadata are returned as initially read from the storage volume.

Abstract: An improved technique involves providing a pair of inode numbers in a directory entry for a file in a directory of a target file system. Along these lines, the target file system writes a first inode number and a second inode number to the directory entry for the file. While the client has not yet remounted its file system, that client may continue to use the file handle provided for the original file system. The data storage system continues to refer to a mapping table that cross-references initial inode numbers to target ones, at some cost in performance. Nevertheless, the client has an option to remount the file system at a time that is convenient to the client, so that the client may obtain a new handle. The client may subsequently use the new file handle to access the file.

Abstract: For enabling a fast reverse lookup of parent directories storing records of hard links to a specified regular file, a file system manager maintains a hard link database (HLDB) storing information for addressing hard links to the regular files having multiple hard links. The HLDB contains a corresponding HLDB file for each regular file having multiple hard links. In a preferred implementation, the inode of each regular file stores the addressing information for one hard link to the regular file, a corresponding HLDB file stores the addressing information for other hard links to the regular file, each HLDB file is a directory accessed by invoking file system manager directory access routines, and the HLDB has a directory tree providing a multi-level hash index for searching the HLDB given the inode number of a regular file.

Abstract: A dataset is recovered after a server reboot while clients access the dataset. In response to the reboot, not-yet-completed transactions in a log are parsed to create, for each of the dataset blocks modified by these active transactions, a respective block replay list of the active transactions that modify the block. Once the block replay lists have been created, clients may access specified blocks of the dataset after on-demand recovery of the specified blocks. The on-demand recovery is concurrent with a background recovery task that replays the replay lists. To accelerate log space recovery, the parsing of the log inserts each replay list into a first-in first-out queue serviced by multiple replay threads. The queue can also be used as the cache writeback queue, so that the cache index is used for lookup of the replay list and the recovery state of a given block.

Abstract: In a file server, a file system is built upon a volume of data storage. The file system includes multiple cylinder groups. Each cylinder group includes file system blocks. The file system blocks include allocated blocks and free blocks. The volume includes slices of storage, and each slice stores at least one of the cylinder groups. A hierarchical slice map has a top level that includes a count of free blocks in the file system, an upper level that includes a count of free blocks in groups of the slices, an intermediate level that includes a count of the free blocks in sub-groups of the slices, and a bottom level that includes a count of the free blocks in each slice of storage. To find a free block for allocation to a file, the slice map hierarchy is searched in a top-down fashion.

Abstract: To recover from partial writes of metadata transactions to a file system log, a redundancy check is added to each log record. Upon re-boot of the file server, the file system is recovered by finding the last valid record in the log, and then replaying records from the log into the on-disk file system. The replay is terminated prior to the last valid record upon reaching any record with a bad redundancy check. A “fsck” is performed on the file system prior to granting client access to the file system only if the log recovery was terminated prior to the last valid record upon reaching a record with a bad redundancy check and the position of the records in the log indicates that this record with a bad redundancy check could not have been written to the log concurrently with the last valid record.