Abstract: Techniques for CPU cache efficiency may include performing concurrent processing, such as for first and second data operations, in a synchronized manner that prevents loading the same data chunk into the CPU cache more than once. Processing may include synchronizing the first and second data operations with respect to a first data chunk to ensure that both the first and second data operation processing has completed prior to proceeding with performing such processing on a second data chunk. The first and second data operations may be any two of deduplication, encryption, and compression, performed inline as part of the data path. In one embodiment, the first and second data operations for the first data chunk may be performed in parallel or sequentially where neither data operation proceeds with another data chunk until processing of the first and second data operations is complete for the first data chunk.

Abstract: A technique for overwriting compressed data tests whether new data compressed with a first compression procedure fits within spaces provided for previous data. If the compressed new data does not fit, the technique compresses the new data using a second compression procedure. Assuming the second compression procedure reduces the compressed size of the new data to fit the available space, the technique stores the new data in the same location as the previous data. In this manner, overwrites can be accommodated in place without the need to create new mapping metadata.

Abstract: A method of operating a data storage system to provision space on a volume on which a set of filesystems rest includes (a) storing data in a set of files of the set of filesystems, files of a first subset being marked as thick files and files of a second subset being marked as thin files, and (b) reserving backing store for the volume. Reserving backing store includes (1) summing, over all the files of the first subset, a block representation of the filesize of each file, yielding a first sum, (2) summing, over all files of the second subset, a number of filesystem blocks of each file that store active file data, yielding a second sum, (3) calculating a reservation size by summing the first and sums, and (4) providing a number of blocks of backing store for the volume equal to at least the calculated reservation size.

Abstract: A method, computer program product, and computer system for receiving, at a computing device, an I/O request directed to a compressed data portion of a file system. A field associated with the compressed data portion may be determined to be invalid. An error may be generated in response to determining that the field associated with the compressed data portion is invalid. The file system may be maintained in an online status while the compressed data portion is invalid.

Abstract: In response to a cache flush event indicating that host data accumulated in a cache of a storage processor of a data storage system is to be flushed to a lower deck file system, an aggregation set of blocks is formed within the cache, and a digest calculation group is selected from within the aggregation set. Hardware vector processing logic is caused to simultaneously calculate crypto-digests from the blocks in the digest calculation group. If one of the resulting crypto-digests matches a previously generated crypto-digest, deduplication is performed that i) causes the lower deck file system to indicate the block of data from which the previously generated crypto-digest was generated and ii) discards the block that corresponds to the matching crypto-digest. Objects required by a digest generation component may be allocated in a just in time manner to avoid having to manage a pool of pre-allocated objects.

Abstract: Techniques for effectively measuring the amount of space saved due to deduplication and selectively turning deduplication off for a filesystem if the savings are too low are provided. The techniques include maintaining both a lower bound count and an upper bound count for block savings due exclusively to deduplication rather than due to snapshots and using an average of the lower and upper bounds to estimate the actual savings due to deduplication in order decide whether to continue operating deduplication or not.

Abstract: A technique for efficiently storing compressed data of a storage object in a data storage includes (a) receiving, in a cache buffer, a number, U, of uncompressed blocks of a uniform size, the uncompressed data blocks received in write requests directed to the storage object; (b) compressing the uncompressed blocks of the cache buffer into respective compressed extents; (c) performing an optimization operation including generating a set of distributions of compressed extents among a plurality of containers and searching the set for a distribution having a minimal total amount of storage taken up by its respective plurality of containers, each container having a respective size equal to a respective integer multiple of the uniform size of the uncompressed data blocks; and (d) storing the compressed extents within a plurality of containers in persistent storage in accordance with the distribution having the minimal total amount of storage taken up by its respective plurality of containers.

Abstract: In a data storage system an internal file system stores client storage objects as respective files. A count of actual free space includes blocks available for allocation to store new file data, and a count of reported free space includes the actual free space as well as blocks of recently deleted file data undergoing deallocation. The reported free space count is used to initiate file system operations requiring allocation of free space, while the actual free space count is used to continue initiated file system operations based on availability of actual free space. Upon a truncation, a reporting update estimates or counts a number of blocks to be freed and adds the number to the reported free space count, and a background update is initiated to deallocate truncated blocks at a paced rate and update the actual free space count as blocks are deallocated.

Abstract: Techniques are described for building, managing, and enforcing nested quota trees in a filesystem of a data storage system using internal filesystem structures. Such integrated, server-based, nested quota trees avoid the management overhead, communication delays, and slowed processing of proxy-based solutions and are thus more efficient and optimal.

Abstract: A method of operating a DSS to provision space on a volume on which a filesystem and its snapshots rest, having thin and thick files. It includes (a) receiving a reservation-altering storage command, (b) calculating a volume liability by summing together (1) a total number of allocated data blocks stored on the volume and (2) a maximum value, over each of the filesystem and the snapshots, of a difference between (A) a count of guaranteed thick blocks for that filesystem or snapshot based on a count received from the filesystem including a sum, over each thick file on the filesystem, of data blocks guaranteed to that file and (B) a number of uncompressed mapped data blocks marked as thick by the filesystem for that filesystem or snapshot, and (c) providing a number of blocks of backing store for the volume equal to at least the calculated volume liability.

Abstract: A technique for managing data storage applies both inline software compression and inline hardware compression in a data storage system, using both types of compression together. The data storage system applies inline software compression for compressing a first set of newly arriving data and applies inline hardware compression for compressing a second set of newly arriving data. Both sets of data are directed to a data object, and the data storage system compresses both sets of data without first storing uncompressed versions thereof in the data object.

Abstract: A technique for managing storage space in a data storage system generates liability values on a per-family basis, with each family including files in the file system that are related to one another by snapping. Each family thus groups together files in the file system that share at least some blocks among one another based on snapshot activities. Distinct files that do not share blocks based on snapping are provided in separate families. The file system leverages the snap-based relationships among family members to produce more accurate estimates of liability than would otherwise be feasible.

Abstract: A method is used in managing storage space in storage systems. A request is received to write data to a logical storage object. A determination is made as to whether the data can be written to the logical storage object in a compressed format. Based on the determination, the request is processed based on a storage insurance value and a storage liability value associated with the logical storage object. The storage insurance value and storage liability value is determined based on the number of uncompressed blocks included in the logical storage object.

Abstract: A method is used in managing I/O requests in file systems. A first and second I/O requests are received. The first I/O request is directed to a first portion of a file and the second I/O request is directed to a second portion of the file. Respective first portions of the first and second I/O requests are performed concurrently. Respective second portions of the first and second I/O requests are performed concurrently. Execution of the second I/O request starts before execution of the first I/O request finishes.

Abstract: A method, computer program product, and computing system for receiving a storage operation request for a logical storage object. A storage insurance value is associated with the logical storage object. The storage operation request is processed to associate a storage liability value with the storage operation request. Whether the storage operation request should be effectuated is determined based, at least in part, upon the storage liability value and the storage insurance value.

Abstract: A technique performs a write operation. The technique involves receiving, after a snapshot of a file system is taken, a write instruction to write new data to a particular portion of the file system. The technique further involves includes reading, in response to the write instruction, metadata to determine whether the new data is a first update of the particular portion since the snapshot was taken. The metadata identifies portions of the file system which have changed since the snapshot was taken. The technique further involves, when the new data is a first update of the particular portion since the snapshot was taken, copying the original data from cache memory (i.e., from the file system buffer cache) to snapshot storage (i.e., to a dedicated save area) and, after the original data is copied, updating the original data in the cache memory with the new data.

Abstract: The version set backup and restore facility responds to a version set backup request by backing up multiple snapshot copies of a storage object concurrently from a storage array to backup storage media. The version set backup and restore facility responds to a version set restore request by restoring a plurality of snapshot copies of the storage object concurrently from the backup storage media to the storage array. The on-tape backup image of the version set includes variable-length extents of data for each of the multiple snapshot copies of the storage object. The variable-length extents of data for each of the snapshot copies of the storage object are grouped together and ordered in the on-tape backup image by increasing or decreasing logical block address.