Abstract: There are disclosed techniques for reporting space savings due to compression in storage systems. In one embodiment, the techniques comprise receiving a request to write data to a data storage system. The techniques also comprise performing a compression of the data associated with the request. The techniques further comprise determining a difference between a number of allocation units needed if no compression of the data associated with the request and a number of allocation units allocated to service the compressed data associated with the request. The techniques still further comprise providing a data reduction attributed to compression based on the difference.

Abstract: Metadata formatting techniques may include: receiving a request to allocate M blocks of a file system; selecting for allocation, using an allocation bitmap, M free blocks of the file system, wherein the M free blocks have corresponding block metadata (BMD) structures included in a BMD group stored in a first block of physical storage; determining whether the BMD group is stored in cache; determining, using the allocation bitmap, whether each BMD structure of the BMD group is unallocated; and responsive to determining the BMD group is not stored in cache and that each BMD structure of the BMD group is unallocated, performing first processing initializing the corresponding BMD structures for the M free blocks to denote the M free blocks are allocated. The first processing may include storing in the cache initialized versions of the corresponding BMD structures for the M free blocks.

Abstract: A data storage system operates to shrink an intermediate logical volume and lower file system upon deletion of a first upper data unit of an upper file system without moving file data on the physical storage devices. Initially, a second upper data unit of the upper file system is identified that is mapped to an end location of the intermediate logical volume. Within the lower file system, a non-end location of the volume file is re-mapped to a first location on the physical storage devices where the data of the end location of the intermediate logical volume is stored. Within the upper file system, the second upper data unit is re-mapped to the non-end location of the intermediate logical volume. Subsequently, the intermediate volume is truncated by removing the portion at the non-end location.

Abstract: Techniques for effectively measuring the amount of space saved due to deduplication and selectively turning deduplication off for a filesystem if the savings are too low are provided. The techniques include maintaining both a lower bound count and an upper bound count for block savings due exclusively to deduplication rather than due to snapshots and using an average of the lower and upper bounds to estimate the actual savings due to deduplication in order decide whether to continue operating deduplication or not.

Abstract: Techniques for improving storage efficiency of a data storage system are provided. The techniques include (a) performing a testing operation to evaluate whether a test condition is satisfied for a filesystem having a plurality of subspaces, each subspace being provisioned with a respective amount of storage, each subspace of the filesystem being dedicated to storing different respective types of filesystem content, wherein the test condition indicates that a first subspace has available space in excess of a threshold percentage, (b) in response to determining that the test condition is satisfied for the filesystem, performing a reduction operation on the first subspace to remove a portion of storage provisioned to that subspace while retaining the removed portion for use by the filesystem, and (c) upon a second subspace reaching a fullness threshold, provisioning the removed portion to the second subspace and storing content of the respective type of the second subspace thereon.

Abstract: An in-line (or foreground) approach to obtaining contiguous ranges of free space in a file system of a data storage system that can select windows having blocks suitable for relocation at a time when one or more blocks within the respective windows are freed or de-allocated. By providing the in-line or foreground approach to obtaining contiguous ranges of free space in a file system, a more efficient determination of windows having blocks suitable for relocation can be achieved, thereby conserving processing resources of the data storage system.

Abstract: A method is used in managing inline data compression in storage systems. A request is received to update data of a data object previously stored in an allocation unit of a segment in a storage system. The segment comprises of a set of allocation units in a compressed format. A determination is made as to whether the updated data of the data object can be stored in the compressed format in the allocation unit of the segment. Based on the determination, the updated data is written in the compressed format to the allocation unit of the segment.

Abstract: A method is used in managing inline data compression in storage systems. A request is received to update data of a data object previously stored in an allocation unit of a segment in a storage system. The segment comprises of a set of allocation units in a compressed format. A determination is made as to whether the updated data of the data object can be stored in the compressed format in the allocation unit of the segment. Based on the determination, a free allocation unit is selected from a list of free allocation units managed by the storage system for writing the updated data in the compressed format to the free allocation unit.

Abstract: An improved technique for managing data storage includes relocating allocated blocks within a range of a physical address space of a file system to free the allocated blocks. The range has a length equal to a length of one or more full stripes of a RAID group that stores content of the file system. In response to receiving data to be written to the file system, the file system arranges the data in the range of contiguous blocks and performs one or more full-stripe writes to write the data to the RAID group.

Abstract: Techniques for shrinking a filesystem backed by a volume identify slices in the volume to be evacuated in order to reach a target size, identify a target endpoint in the volume, and evacuate identified slices to available locations prior to the target endpoint. The same data is typically not moved from slice to slice multiple times.

Abstract: Embodiments are directed to techniques for allowing write operations to proceed on units smaller than a block. Merely decreasing the block size is not desirable, however, since many files are written in large chunks at once, and larger block sizes can increase speed and decrease the amount of overhead metadata required. Therefore, in order to maintain large block sizes while still obtaining the benefit of being able to make small writes when necessary, blocks may be divided into sub-blocks. Unaligned writes that do not fill up an entire block may be segregated and stored separately from aligned writes, the unaligned writes having finer granularity. These techniques may result in faster operation for unaligned writes. They also especially benefit systems using compression because they allow efficiently partially overwriting compressed blocks.

Abstract: A technique for managing storage space in a data storage system generates liability values on a per-family basis, with each family including files in the file system that are related to one another by snapping. Each family thus groups together files in the file system that share at least some blocks among one another based on snapshot activities. Distinct files that do not share blocks based on snapping are provided in separate families. The file system leverages the snap-based relationships among family members to produce more accurate estimates of liability than would otherwise be feasible.

Abstract: A technique for managing storage space in a file system applies reference weights to compressed storage extents stored in multi-block segments. Each time a reference weight for a compressed storage extent in a segment changes to a value that indicates that the storage extent is no longer in use, a file system manager increments a freed-fragment counter provided for a range of the file system that includes the segment. The file system manager then determines whether to scavenge for free space in that range of the file system based at least in part on a count of the free-fragment counter.

Abstract: A data storage system tracks liability and insurance for an internal file system, liability being a number of slices needed to store file system data, insurance being a number of slices allowed to be consumed. A reserve of un-provisioned insurance is maintained from which slices are provisioned to the file system for use in storing file system data without immediate requirement for increasing the insurance. Slices are provisioned to the file system from the reserve of un-provisioned insurance based on a window criteria and a space criteria, the window criteria being that a number of free windows is less than a free window threshold, the space criteria being that a number of currently provisioned slices is less than a maximum allowed slices, which includes an overprovisioning factor applied to the primary file size to allow for growth of the primary file without immediate requirement for increasing provisioned insurance.

Abstract: Described are techniques for creating windows of free blocks in a file system selected in accordance with trigger conditions. A first slice is selected, in accordance with slice selection criteria, from a plurality of slices of storage provisioned for use by the file system. First processing is performed on the first slice that creates at least one window of free blocks in the first slice for use by the file system. It is determined, in accordance with termination criteria, whether to process one or more additional slices of the file system to create additional windows of free blocks for use by the file system. Such processing to create free windows may be subject to various limits of resource consumption.

Abstract: A technique for managing block allocation in data storage systems involve providing multiple ranges of contiguous logical addresses (logical windows) in a logical address space of a file and allocating contiguous ranges of physical addresses (physical windows) in a file system to respective logical windows in response to requests to write data to logical addresses within the logical windows. When allocating blocks from one or more streams, each logical window to which writes are directed is mapped to a respective physical window, and each logical address in the logical window is mapped to a physical address in the physical window.

Abstract: A technique for storage allocation provides a first cursor and a second cursor from which to allocate blocks within a physical address space of a file system. The file system uses the first cursor for allocating blocks for writes directed to sequential logical addresses and uses the second cursor for writes directed to random (non-sequential) logical addresses.

Abstract: A method of operating a data storage system includes writing the file system data as sequential data and non-sequential data to a storage volume, the sequential data being stored in windows each having a predetermined number of consecutive data blocks and being allocated dynamically as the sequential data is written. The method includes maintaining and using a window cache to identify existing windows for storing respective newly written sequential file system data in sequence with respective earlier-written file system data for which the existing windows were previously allocated, the window cache including a set of entries indexed by an identifier of (1) a file of the file system and (2) a window-size region of the file to which sequential data is being written, the entries including respective physical window addresses identifying respective ones of the existing windows and being obtained by lookup operations using respective values of the identifier.

Abstract: A technique for managing storage tiering in a data storage apparatus operates at a fine level of granularity in response to write requests as blocks are allocated for writing. As write requests arrive or are otherwise processed by the data storage apparatus, the data storage apparatus identifies a quality of service to be provided for satisfying each write request and allocates a set of blocks from storage devices of storage tiers that provide the identified quality of service. The data storage apparatus may then store the information specified in the write request in the newly allocated blocks.

Abstract: A technique for managing a file system includes dividing the addressable space of a file system into multiple subspaces: a data subspace for user data; an inode subspace for inodes describing the user data; and an MDB (metadata block) subspace for indirect blocks and/or directory structures pertaining to the user data. In response to storage requests to write user files to the file system, the file system stores user data of the files in the data subspace, stores inodes pertaining to the user files in the inode subspace, and stores indirect blocks and/or directories in the MDB subspace.