Abstract: A technique for managing replication of VSPs (Virtualized Storage Processors) proceeds on a per-VSP basis by (i) identifying the data objects associated with a VSP, (ii) establishing a common set of replication settings across all of the data objects associated with the VSP, and (iii) replicating the VSP by replicating each of the identified data objects associated with the VSP in accordance with the common set of replication settings established across all of the data objects associated with the VSP. The technique avoids the need for administrators to separately configure and manage replication on large numbers of individual data objects and thus reduces administrative burdens.

Abstract: A technique performs virtual storage processor (VSP) load balancing. The technique involves receiving a VSP move command to load balance a particular VSP from a source physical storage processor to a destination physical storage processor. The technique further involves relinquishing, by the source physical storage processor, access to a set of VSP definitions that define the particular VSP. The technique further involves obtaining, by the destination physical storage processor, access to the set of VSP definitions that define the particular VSP, the particular VSP being load balanced from the source physical storage processor to the destination physical storage processor upon the destination physical storage processor obtaining access to the set of VSP definitions that define the particular VSP.

Abstract: An improved technique for a data storage apparatus that combines both block-based and file-based functionality in a unified data path architecture. The improved technique brings together IO processing of block-based storage systems and file-based storage systems by expressing both block-based objects (e.g., LUNs) and file-based objects (e.g., file systems) in the form of files. These files are parts of an underlying, internal set of file systems, which are stored on a set of storage units served by a storage pool. Because block and file-based objects are all expressed as files of this set of file systems, a common set of services can be applied across block-based and file-based objects. Also, storage units released by any file or files of the underlying, internal set of file systems can be reused by any other file or files, regardless of whether the files represent block-based objects or file-based objects. Inefficiencies of stranded storage are thus greatly reduced or eliminated altogether.

Abstract: A method, computer program product, and computer system for instantiating, by a computing device, a slice-object associated with a slice when the slice-object is accessed. The slice-object is released to a slice object cache when accessing is complete. It is determined whether the slice is accessed within a threshold period of time. If the slice is accessed within the threshold period of time, the slice-object is retrieved from the slice-object cache. If the slice is not accessed within the threshold period of time, memory used for the slice-object is released.

Abstract: A technique for protecting host data using, for example, snaps, asynchronous replication, and/or synchronous replication, includes storing both block-based objects and file-based objects in a common form—as files. With both block-based objects and file-based objects represented as the same type of underlying objects, data protection of both block-based and file-based objects is accomplished using a single set of data protection technologies, which are configured to perform data protection operations on files.

Abstract: A technique manages data within a data storage apparatus having a cache of volatile memory and non-volatile storage. The technique involves receiving a set of write commands to write data to the non-volatile storage. The technique further involves acknowledging, in a write-back caching manner, completion of the set of write commands after the data reaches the cache of volatile memory and prior to the data reaching the non-volatile storage. The data includes host data as well as critical metadata. The technique further involves saving, after acknowledging completion of the set of write commands, the data from the cache of volatile memory to the non-volatile storage, the critical metadata being saved from the cache of volatile memory to the non-volatile storage at a quicker rate than that of the host data (i.e., accelerated synchronization of the critical metadata).

Abstract: A slice consistency check method is presented including, for each slice of a plurality of slices (a) obtaining a first vote for ownership of that slice according to a filesystem manager configured to manage a set of filesystems stored on a set of sparse metavolumes, (b) obtaining a second vote for the ownership of that slice according to a first driver, the first driver being independent of the filesystem manager, (c) obtaining a third vote for the ownership of that slice according to a second driver, the second driver being independent of the filesystem manager and the first driver, and (d) assigning ownership of that slice based on a majority vote of the first vote for the ownership of that slice, the second vote for the ownership of that slice, and the third vote for the ownership of that slice.

Abstract: A data storage system includes storage devices and a processing subsystem executing software forming a lower-deck file system and an upper-deck file system. The lower-deck file system presents a volume file from which storage is provided to the upper-deck file system, and units of storage of the storage devices are reserved to the upper-deck file system. The volume file is fully provisioned based on the reserved units of storage so that an out-of-space condition is not presented to the upper-deck file system. Reserved units of storage are dynamically added to the volume file to increase its allocated size in response to demand from the upper-deck file system, and a direct operating mode is used in which a block address in the lower-deck file system is obtained by a predetermined calculation from a corresponding block address in the volume file.

Abstract: A data storage system includes storage devices and a processing subsystem executing software forming a lower-deck file system and an upper-deck file system. The lower-deck file system presents a volume file from which storage is provided to the upper-deck file system, and units of storage of the storage devices are allocable to the upper-deck file system but not reserved to it. The volume file is thinly provisioned, and additional units of storage are added dynamically to increase its allocated size in response to demand from the upper-deck file system. The lower-deck file system operates in a mapped mode in which a block address in the lower-deck file system is obtained by a generally arbitrary mapping (e.g., an Inode/IB tree) of a block address in the volume file.

Abstract: A data storage system employs a multi-level stack including a lower-deck file system and an upper-deck file system providing a file system interface to a file system consumer. The lower-deck file system presents a volume file to provide storage to the upper-deck file system. Storage is reserved to the upper-deck file system, and the volume file is fully provisioned so an out-of-space condition is not presented to the upper-deck file system. In an initial direct operating mode a block address in the lower-deck file system is obtained by a calculation from a corresponding block address in the volume file. Snapshots of a primary volume file are created and stored in respective snapshot volume files. The lower-deck file system transitions into a mapped mode using generally arbitrary mappings of block addresses of the primary volume file and snapshot volume files to obtain corresponding block addresses in the lower-deck file system.

Abstract: A file server has a data compression facility and also a snapshot copy facility or a data de-duplication facility that shares data blocks among files. Compression of the file data on a file system block basis leads to a problem of partially used file system data blocks when the data blocks are shared among files. This problem is solved by partial block allocation so that file system data blocks are shared among files that do not share identical data. Block pointers in the file mapping metadata point to virtual blocks representing the compressed data blocks, and associated virtual block metadata identifies portions of file system data blocks that store the compressed data. For example, a portion of a file system data block is identified by a sector bitmap, and the virtual block metadata also includes a reference count to indicate sharing of a compressed data block among files.

Abstract: An improved technique involves providing read-only access to the file system while FSCK is running. After a storage processor detects corrupted data in the file system, the storage processor brings the file system offline and provides FSCK with read-only access to the file system. The storage processor then begins a repair process on the disk by starting to run FSCK, which writes corrupted blocks to a secondary storage device. Upon receiving a request from a user for access to a file in the file system, the storage processor mounts the file system with read-only access. Upon completion of FSCK, the storage processor brings the file system offline, applies the corrected blocks from secondary storage to the blocks in primary storage, and brings the file system online with full read and write access.

Abstract: A method is used in managing global data caches for file systems. Space is allocated in a volatile memory of a data storage system to a global data cache that is configured to store a set of data objects for a plurality of different file systems. The set of data objects is accessed by the plurality of different file systems. Contents of a file of a file system are stored in a data object in the global data cache upon receiving a write I/O request for the file. A copy of the data object and information for the data object are stored in a persistent journal that is stored in a non-volatile memory of the data storage system. Contents of the file are updated on a storage device based on the data object stored in the global data cache and information stored in the persistent journal.

Abstract: An improved technique involves providing a pair of inode numbers in a directory entry for a file in a directory of a target file system. Along these lines, the target file system writes a first inode number and a second inode number to the directory entry for the file. While the client has not yet remounted its file system, that client may continue to use the file handle provided for the original file system. The data storage system continues to refer to a mapping table that cross-references initial inode numbers to target ones, at some cost in performance. Nevertheless, the client has an option to remount the file system at a time that is convenient to the client, so that the client may obtain a new handle. The client may subsequently use the new file handle to access the file.

Abstract: A technique forms contents of a compression container from a standalone container file system stored on the same data storage array on which the compression container is stored. Such a standalone container includes a container file which is formed by standalone blocks. The standalone blocks store host data and which are hierarchically related to each other. Upon receiving a compression command, the storage processor carries out a set of compression operations which include creating and storing the compression container in the data storage array. The compression container includes container files which are formed by compression blocks storing the host data and which are hierarchically related to each other.

Abstract: In a data storage system, a layered operating stack includes a file system layer, a file cache layer, and a unified cache layer including page descriptors each capable of referencing a data block stored in either/both DRAM or/and flash memory. A reference structure associates file references with corresponding page descriptors. The unified cache layer manages use of the memory for caching data blocks of files being written at the file system layer, including allocation, access, and eviction/destaging. Allocation establishes associative references in the reference structure, and access uses the reference structure to locate cached data blocks based on file references. Eviction/destaging relocates cached data blocks as necessary and includes first-level destaging from DRAM to flash and second-level destaging flash to physical storage. The first-level destaging maintains existing associative references and modifies page descriptors to add references to the flash.

Abstract: An improved technique for a data storage apparatus that combines both block-based and file-based functionality in a unified data path architecture. The improved technique brings together IO processing of block-based storage systems and file-based storage systems by expressing both block-based objects (e.g., LUNs) and file-based objects (e.g., host file systems) in the form of files. These files are parts of an underlying set of internal file systems stored on a set of storage units served by a storage pool. Because block and file-based objects are all expressed as files of this set of internal file systems, a common set of services can be applied across block-based and file-based objects. In particular, enhanced data services such as compression, automated storage tiering and deduplication are provided across both types of object using one set of common mechanisms.

Abstract: For enabling a fast reverse lookup of parent directories storing records of hard links to a specified regular file, a file system manager maintains a hard link database (HLDB) storing information for addressing hard links to the regular files having multiple hard links. The HLDB contains a corresponding HLDB file for each regular file having multiple hard links. In a preferred implementation, the inode of each regular file stores the addressing information for one hard link to the regular file, a corresponding HLDB file stores the addressing information for other hard links to the regular file, each HLDB file is a directory accessed by invoking file system manager directory access routines, and the HLDB has a directory tree providing a multi-level hash index for searching the HLDB given the inode number of a regular file.

Abstract: A non-disruptive upgrade technique involves, with (i) a first SP using first virtual SPs (VSPs) defining first environments for first host file systems, (ii) a second SP using second VSPs defining second environments for second host file systems, and (iii) an initial software version installed on each SP, processing host I/Os on the host file systems by the SPs. The technique further involves moving the first VSPs from the first SP to the second SP to provision the second SP to process host I/Os concurrently on the first and second host file systems using the first and second VSPs. The technique further involves, after moving the first VSPs from the first SP to the second SP and while the second SP processes host I/Os concurrently on the first and second host file systems using the first and second VSPs, installing a new backwards compatible software version on the first SP.

Abstract: A technique for processing IO requests in a data storage system provides active-active access to pooled data objects from multiple storage processors. Data objects, which may include both block-based objects and file-based objects, are realized in the form of respective files stored in a set of clustered internal file systems of the data storage system. For providing active-active access to a data object, each of multiple storage processors operates such a clustered file system, which coordinates with a clustered file system on each of the other storage processors to present a consistent file system image having a single namespace across all such storage processors. On each storage processor, the clustered file system is built upon a clustered storage pool, which maintains consistency with each of the clustered storage pools running on each of the other storage processors, to present a consistent image of storage allocation across all storage processors.