Abstract: An aspect of space accounting includes providing a counter for each of a plurality of key/value pairs for storage system operations. The key uniquely identifies a logical storage unit on which operations associated with the key are performed, the value identifies an amount of data usage with respect to the operations performed on the logical storage unit, and the counter tracks a type of the operations. An aspect also includes tracking, via corresponding counters, updates to a read/write (RW) tablet associated with a Cuckoo tree. The tracking includes updating a filter to include instances of key/value pairs and the RW tablet in which the pairs are stored. An aspect further includes tracking a plurality of read-only (RO) tablets associated with the Cuckoo tree. The tracking includes updating the filter to include instances of key/value pairs and one or more of the RO tablets in which the pairs are stored.

Abstract: Techniques for performing online recovery of space accounting metadata. The techniques include reestablishing a plurality of counters for tracking amounts of physical storage space committed and/or unique to the volumes and/or volume families. The techniques include, while reestablishing the counters, conducting a tree walk through a mapping hierarchy of each volume and/or volume family, marking starting and ending points through the mapping hierarchy, and monitoring a logical offset from the starting point during the tree walk. Upon receipt of an IO request, the techniques determine whether the IO request and space accounting metadata recovery activities are attempting to access the same metadata based on the logical offset relative to the starting point, and update the counters and/or temporary versions of the counters, as appropriate. By providing such techniques, metadata inconsistencies can be addressed while allowing users of the data storage system full or at least partial access to their data.

Abstract: Techniques for performing space accounting for volume families. The techniques include maintaining two counters for each volume family, including a first counter that tracks a first amount of physical space allocated to volumes in the volume family based on the number of pages written to the volumes, and a second counter that tracks a second amount of physical space owned by the volume family, and maintaining a third counter for each volume in each branch of the volume family that tracks a number of pages among a total number of pages written to the volume that were overwritten in the immediate sequential next volume in the same branch as the volume. By maintaining, for each volume family, the first counter and the second counter, and, for each volume in the volume family, the third counter, space accounting metrics can be obtained that allow data storage activities to be performed efficiently.

Abstract: A method of repairing an indirect addressing structure of a file system damaged by corruption of a virtual data block (VDB) mapping data stored in corresponding physical data blocks (PDBs) includes scanning PDB descriptors to identify PDBs storing data mapped by the corrupted VDB, where each identified PDB includes a set of PDB entries each having a backward pointer identifying a corresponding VDB entry of a corresponding VDB. The identified PDBs are scanned to identify PDB entries whose backward pointers refer to VDB entries of the corrupted VDB, then a replacement VDB is created by (1) for each of the identified PDB entries, recreating a corresponding VDB entry including a forward pointer to the identified PDB entry, and (2) incorporating the recreated VDB entries into the replacement VDB. The replacement VDB is then incorporated into the indirect addressing structure in place of the corrupted VDB.

Abstract: A technique for repairing an indirect addressing structure of a file system damaged by corruption of a mid-level mapping (MID) page includes scanning selected leaf pages to identify leaf pages associated with the corrupted MID page, then recreating the MID page by recreating pointers to the identified leaf pages. The scanning includes (1) based on an association of groups of leaf pages with corresponding sets of families of storage objects, scanning the leaf pages of only those groups of leaf pages associated with the family of storage objects for the corrupted MID page. The scanning further includes (2) performing a two-pass process including first identifying all leaf pages for the logical offset range of the corrupted MID page and then pruning those identified leaf pages that are reachable via non-corrupted MID pages, yielding the leaf pages for the corrupted MID page only, usable to recreate the corrupted MID page.

Abstract: An in-line (or foreground) approach to obtaining contiguous ranges of free space in a file system of a data storage system that can select windows having blocks suitable for relocation at a time when one or more blocks within the respective windows are freed or de-allocated. By providing the in-line or foreground approach to obtaining contiguous ranges of free space in a file system, a more efficient determination of windows having blocks suitable for relocation can be achieved, thereby conserving processing resources of the data storage system.

Abstract: An aspect of space accounting includes providing a counter for each of a plurality of key/value pairs for storage system operations. The key uniquely identifies a logical storage unit on which operations associated with the key are performed, the value identifies an amount of data usage with respect to the operations performed on the logical storage unit, and the counter tracks a type of the operations. An aspect also includes tracking, via corresponding counters, updates to a read/write (RW) tablet associated with a Cuckoo tree. The tracking includes updating a filter to include instances of key/value pairs and the RW tablet in which the pairs are stored. An aspect further includes tracking a plurality of read-only (RO) tablets associated with the Cuckoo tree. The tracking includes updating the filter to include instances of key/value pairs and one or more of the RO tablets in which the pairs are stored.

Abstract: A method is used in managing migration of virtual file servers. The method migrates a virtual file server from a source storage processor to a destination storage processor in a storage system. The storage system includes the source and the destination storage processors. The virtual file server comprises a root file system, a configuration file system, and a set of user file systems. The method enables concurrent access to the root file system from both source and destination storage processors during the migration until the set of user file systems is migrated from the source storage processor to the destination storage processor.

Abstract: A method of repairing an indirect addressing structure of a file system damaged by corruption of a virtual data block (VDB) mapping data stored in corresponding physical data blocks (PDBs) includes scanning PDB descriptors to identify PDBs storing data mapped by the corrupted VDB, where each identified PDB includes a set of PDB entries each having a backward pointer identifying a corresponding VDB entry of a corresponding VDB. The identified PDBs are scanned to identify PDB entries whose backward pointers refer to VDB entries of the corrupted VDB, then a replacement VDB is created by (1) for each of the identified PDB entries, recreating a corresponding VDB entry including a forward pointer to the identified PDB entry, and (2) incorporating the recreated VDB entries into the replacement VDB. The replacement VDB is then incorporated into the indirect addressing structure in place of the corrupted VDB.

Abstract: A technique for repairing an indirect addressing structure of a file system damaged by corruption of a mid-level mapping (MID) page includes scanning selected leaf pages to identify leaf pages associated with the corrupted MID page, then recreating the MID page by recreating pointers to the identified leaf pages. The scanning includes (1) based on an association of groups of leaf pages with corresponding sets of families of storage objects, scanning the leaf pages of only those groups of leaf pages associated with the family of storage objects for the corrupted MID page. The scanning further includes (2) performing a two-pass process including first identifying all leaf pages for the logical offset range of the corrupted MID page and then pruning those identified leaf pages that are reachable via non-corrupted MID pages, yielding the leaf pages for the corrupted MID page only, usable to recreate the corrupted MID page.

Abstract: Techniques for tracking pages that are unshared among snapshot volumes and their parent volume in a volume family. The techniques include maintaining a first counter that tracks a first number (“SnapOwnedCount”) of pages among a total number of pages written to a volume that were overwritten in the immediate sequential next volume in the same branch as the volume, and a second counter that tracks a second number (“UnsharedWithParentCount”) of pages among the total number of pages written to the volume that were overwritten in any sequential next volume in the same branch as the volume. By maintaining, for each volume in a volume family, a first counter and a second counter for tracking the SnapOwnedCount and the UnsharedWithParentCount, respectively, data storage recovery can be performed more effectively, allowing selected volumes and/or portions thereof to be deleted while maintaining accurate space accounting information.

Abstract: Techniques for performing space accounting for volume families. The techniques include maintaining two counters for each volume family, including a first counter that tracks a first amount of physical space allocated to volumes in the volume family based on the number of pages written to the volumes, and a second counter that tracks a second amount of physical space owned by the volume family, and maintaining a third counter for each volume in each branch of the volume family that tracks a number of pages among a total number of pages written to the volume that were overwritten in the immediate sequential next volume in the same branch as the volume. By maintaining, for each volume family, the first counter and the second counter, and, for each volume in the volume family, the third counter, space accounting metrics can be obtained that allow data storage activities to be performed efficiently.

Abstract: Techniques for tracking pages that are unshared among snapshot volumes and their parent volume in a volume family. The techniques include maintaining a first counter that tracks a first number (“SnapOwnedCount”) of pages among a total number of pages written to a volume that were overwritten in the immediate sequential next volume in the same branch as the volume, and a second counter that tracks a second number (“UnsharedWithParentCount”) of pages among the total number of pages written to the volume that were overwritten in any sequential next volume in the same branch as the volume. By maintaining, for each volume in a volume family, a first counter and a second counter for tracking the SnapOwnedCount and the UnsharedWithParentCount, respectively, data storage recovery can be performed more effectively, allowing selected volumes and/or portions thereof to be deleted while maintaining accurate space accounting information.

Abstract: Techniques for performing online recovery of space accounting metadata. The techniques include reestablishing a plurality of counters for tracking amounts of physical storage space committed and/or unique to the volumes and/or volume families. The techniques include, while reestablishing the counters, conducting a tree walk through a mapping hierarchy of each volume and/or volume family, marking starting and ending points through the mapping hierarchy, and monitoring a logical offset from the starting point during the tree walk. Upon receipt of an IO request, the techniques determine whether the IO request and space accounting metadata recovery activities are attempting to access the same metadata based on the logical offset relative to the starting point, and update the counters and/or temporary versions of the counters, as appropriate. By providing such techniques, metadata inconsistencies can be addressed while allowing users of the data storage system full or at least partial access to their data.

Abstract: A method is used in managing migration of virtual file servers. The method migrates a virtual file server from a source storage processor to a destination storage processor in a storage system. The storage system includes the source and the destination storage processors. The virtual file server comprises a root file system, a configuration file system, and a set of user file systems. The method enables concurrent access to the root file system from both source and destination storage processors during the migration until the set of user file systems is migrated from the source storage processor to the destination storage processor.

Abstract: An in-line (or foreground) approach to obtaining contiguous ranges of free space in a file system of a data storage system that can select windows having blocks suitable for relocation at a time when one or more blocks within the respective windows are freed or de-allocated. By providing the in-line or foreground approach to obtaining contiguous ranges of free space in a file system, a more efficient determination of windows having blocks suitable for relocation can be achieved, thereby conserving processing resources of the data storage system.

Abstract: A method is used in managing migration of virtual file servers. The method migrates a virtual file server from a source storage processor to a destination storage processor in a storage system. The storage system includes the source and the destination storage processors. The virtual file server comprises a root file system, a configuration file system, and a set of user file systems. The method enables concurrent access to the root file system from both source and destination storage processors during the migration until the set of user file systems is migrated from the source storage processor to the destination storage processor.

Abstract: A data storage system tracks liability and insurance for an internal file system, liability being a number of slices needed to store file system data, insurance being a number of slices allowed to be consumed. A reserve of un-provisioned insurance is maintained from which slices are provisioned to the file system for use in storing file system data without immediate requirement for increasing the insurance. Slices are provisioned to the file system from the reserve of un-provisioned insurance based on a window criteria and a space criteria, the window criteria being that a number of free windows is less than a free window threshold, the space criteria being that a number of currently provisioned slices is less than a maximum allowed slices, which includes an overprovisioning factor applied to the primary file size to allow for growth of the primary file without immediate requirement for increasing provisioned insurance.

Abstract: Described are techniques for creating windows of free blocks in a file system selected in accordance with trigger conditions. A first slice is selected, in accordance with slice selection criteria, from a plurality of slices of storage provisioned for use by the file system. First processing is performed on the first slice that creates at least one window of free blocks in the first slice for use by the file system. It is determined, in accordance with termination criteria, whether to process one or more additional slices of the file system to create additional windows of free blocks for use by the file system. Such processing to create free windows may be subject to various limits of resource consumption.

Abstract: Techniques are presented for keeping only the most commonly-accessed snapshots mounted at any given time, while providing for fast mounting of unmounted snapshots. Techniques are also provided for maintaining separate databases for the snapshots of each primary filesystem in order to minimize locking latency.