Abstract: Techniques for performing processing to recover metadata may include: shadow top structures, and performing processing that uses the shadow top structures to recover information for an index node associated with an object of a file system having a file system logical address space. One of the shadow top structures is created for each metadata (MD) top node of a MD mapping structure used to determine storage locations of data stored at corresponding logical addresses in the file system logical address space. Each MD top node is used in determining storage locations for a specified subrange of logical addresses of the file system logical address space. Each shadow top structure corresponding to a MD top node describes each file system object mapped to a logical address included in the specified subrange of logical addresses of the file system address space associated with the corresponding MD top node.

Abstract: Techniques for handling pattern identifiers in a data storage system. By replacing a block pointer with a pattern identifier, the techniques can identify a data block (or an indirect data block) as a bad block, without resorting to the use of a separate flag or bad block (BB) bit in per-block metadata (e.g., a mapping pointer) of the data block. The techniques can also avoid waste of valuable metadata space by using pattern identifiers at various levels of a mapping tree, leveraging pointer granularity at lower levels, mid-levels, and progressively higher levels of the mapping tree.

Abstract: In response to detecting that at least one snapshot group identifier has been allocated that has a maximum snapshot group identifier value, snapshot group identifiers are allocated to each newly created snapshot group by finding a previously allocated snapshot group identifier that is currently unused, and allocating the previously allocated snapshot group identifier that is currently unused to the newly created snapshot group.

Abstract: A technique for managing data storage places a data object in a first logical extent within a logical address space of a storage system and places a set of attributes assigned to that data object in a second logical extent within the same logical address space of the storage system. In response to a request to generate a second version of the data object, such as a snap, clone, or replica, for example, the first and second logical extents are duplicated atomically, thereby preserving a second version of the attributes along with the second version of the data object.

Abstract: A method of protecting a filesystem against metadata corruption while maintaining high performance is provided. The method includes (a) receiving a storage command directing a computing device to store a block of file data to the filesystem and (b) in response: (1) writing the block to a data portion spanning a set of drives using a single-fault tolerant level of RAID, wherein writing the block includes performing exactly two write operations to the set of drives; and (2) writing filesystem metadata associated with the block to a metadata portion spanning a subset of the set of drives using a double-fault tolerant data redundancy scheme different than the single-fault tolerant level of RAID, the subset including at least three drives, the data and metadata portions being non-overlapping, wherein writing the metadata includes performing at least three write operations to the subset of the set of drives.

Abstract: A computer has a secondary storage device with a filesystem for persistent storage of application data, the filesystem having filesystem objects including files, directories, and group objects such as consistency groups. The group objects specify respective groups of the filesystem objects on which certain filesystem group operations act as a unit. The filesystem also has filesystem routines for performing filesystem operations including single-object operations and group operations. A single-object operation acts upon a specified user file or directory independently (e.g., opening or reading from a file), while a group operation acts upon a respective set of filesystem objects grouped together by a specified group object.

Abstract: A method is used in managing recovery of file systems. A determination is made during recovery of a file system as to whether information required for recovering the file system is missing. Based on the determination, storage for a slice is provisioned. The slice is updated with the information required for recovering the file system. Recovery of the file system is performed.

Abstract: A method is used in managing file deletions in storage systems. A request to delete a portion of a file of a file system is received. A first indirect block is associated with the portion of the file and includes a set of file system blocks. A second indirect block associated with a replica of the file is identified. The first and second indirect blocks share a subset of the set of file system blocks. The replica of the file represents a state of the file at a particular prior point in time. Metadata of the second indirect block is updated indicating that the subset of the set of file system blocks are no longer shared between the first and second indirect blocks.

Abstract: A method of protecting a filesystem against metadata corruption while maintaining high performance is provided. The method includes (a) receiving a storage command directing a computing device to store a block of file data to the filesystem and (b) in response: (1) writing the block to a data portion spanning a set of drives using a single-fault tolerant level of RAID, wherein writing the block includes performing exactly two write operations to the set of drives; and (2) writing filesystem metadata associated with the block to a metadata portion spanning a subset of the set of drives using a double-fault tolerant data redundancy scheme different than the single-fault tolerant level of RAID, the subset including at least three drives, the data and metadata portions being non-overlapping, wherein writing the metadata includes performing at least three write operations to the subset of the set of drives.

Abstract: Techniques for preserving locality of storage accesses to copies of storage objects in a cluster of appliances. Mapping metadata is created indicating regions in a copy of a storage object that are mapped to local storage allocated from a target appliance. Read and write operations to regions of the copy that are mapped to storage in the target appliance are processed within the target appliance. Write operations to regions that are not mapped to storage in the target appliance cause storage to be allocated to the copy in the target appliance, mapped to the regions indicated by the write operations, and used to store the write data. Read operations to regions that are not mapped to storage in the target appliance are performed on an original storage object located on a source appliance, from which the copy of the storage object was made.

Abstract: A method is used in managing data inconsistencies in file systems. A data inconsistency is detected in a file system data block of a file system. The file system includes a set of file system data blocks. Information regarding the data inconsistency is provided to a client of the file system. Based on the information, the client is enabled to determine an action for recovering from the data inconsistency.

Abstract: Described are techniques for processing data. Hash values for the data portions of a logical device are determined. Each data portion has a corresponding one of the hash values. It is determined whether a first of the hash values associated with a first of the data portions is equal to a hash value of an implied data value. If the first hash value is equal to the hash value of the implied data value, first processing is performed including determining whether the first data portion has a current data value equal to the implied data value, and responsive to determining that the first data portion has a current data value equal to the implied data value, performing second processing including updating first metadata for the first data portion to denote that the first data portion has the implied data value. The first data portion is not associated with allocated storage.

Abstract: A method is used in managing truncation of files of file systems. A request is received to delete a portion of a file of a file system. A replica of the file is created. The replica represents a state of the file at a particular prior point in time. The replica shares a set of file system blocks of the file with the file. The portion of the file is deleted by updating metadata of the file. The replica of the file is asynchronously deleted in background by de-allocating the set of file system blocks.

Abstract: A method is used in managing truncation of files of file systems. A request is received to delete a portion of a file of a file system. A set of extents associated with the portion of the file is determined where the file has been identified for deletion. A replica of the set of extents is created. The replica represents a state of the set of extents at a particular prior point in time. The portion of the file is deleted by updating metadata of the file. The replica is asynchronously deleted in background.

Abstract: A mapping apparatus comprises a mapper that translates from an input key to an output key in one or more storage devices. A pre-mapper for processing update operations comprises a plurality of mapping tablets and an in-memory look-up filter to identify a given mapping table storing a given input key. The mapping tablets comprise at least one dynamic in-memory ingress tablet and a plurality of persisted frozen tablets. For a given update operation, a first entry is added to one dynamic in-memory ingress tablet comprising the input key for the given update operation and a corresponding output key where data for the given update operation is stored; and a second entry is added to the look-up filter comprising the input key of the first entry and an identifier of the dynamic in-memory ingress tablet storing the first entry for the given update operation. The dynamic in-memory ingress tablet is persisted as a persisted frozen tablet.

Abstract: In one embodiment, a method is performed by a computing device configured to manage a filesystem stored on persistent storage. The method includes (a) receiving a command to delete a range of a file of the filesystem, (b) removing blocks of the range from being accessible to the filesystem as part of the file, (c) overwriting a block of the range using a secure erasure technique, and (d) subsequent to overwriting the block, freeing the block for re-use by the filesystem. A data storage system, apparatus, and computer program product for performing a similar method are also provided.

Abstract: A technique for managing metadata in a data storage system designates block pointers as either sources or copies, where sources contribute to reference counts of pointed-to structures but copies do not. The technique maintains parent-child relationships between parent BPSs (block pointer sets) and child BPSs, where each BPS includes an array of block pointers. Each child BPS is created as a copy of a parent BPS and has block pointers initially designated as copies. The technique performs a metadata-merge operation to merge the block pointers of the parent BPS into those of a child BPS by promoting attributes of block pointers in the child BPS from copy to source, avoiding any need to perform reference count updates on structures pointed to by promoted block pointers.

Abstract: Described are methods, systems, and apparatus, including computer program products for de-duplicating access control lists (ACLs). A first ACL associated with a first computer file is received. A first checksum based at least in part on the first ACL is calculated. One or more directory entries based on the first checksum are retrieved from a de-duplication directory, wherein each directory entry of the one or more directory entries comprises a reference to an ACL and a name comprising the first checksum. A directory entry of the one or more directory entries is identified that references a second ACL that specifies the same permissions as the first ACL. A reference to the second ACL of the directory entry is added to the first computer file.

Abstract: A method is provided for storing files in a filesystem of a data storage system. The method includes (a) storing data of each file of a set of files of the filesystem in a set of data allocation units (AUs) of the filesystem on the data storage system, each data AU of the set of data AUs having a common data AU size, and (b) storing metadata pertaining to each file of the set of files in a set of metadata AUs of the filesystem on the data storage system, each metadata AU of the set of metadata AUs having a common metadata AU size, the common data AU size being larger than the common metadata AU size. A computerized apparatus and a computer program product are also provided for performing a method similar to that described above.

Abstract: A technique operable in a data storage system performs initialization of storage structures in multiple stages and allows hosts to request access to a data object even when some of the storage structures supporting the data object have not been initialized. Rather than initializing all storage structures for a data object before allowing host access, a storage processor instead initializes only a protocol stub structure. The protocol stub structure is configured to receive protocol requests from hosts and to initiate further processing should a storage request in that protocol be received. The further processing includes, in response to receiving a protocol request directed to a host-accessible data object, identifying storage structures involved in processing that protocol request and directing the storage processor to initialize those storage structures.