Abstract: A technique for managing write requests in a data storage system checks whether newly-arriving data match previously-stored data that have been recorded in a deduplication database. If a match is found, the technique compares mapping metadata for the newly-arriving data with mapping metadata for the matching data. If both sets of metadata point to the same storage location, then the newly-arriving data is a same-data write and a new write to disk is avoided.

Abstract: A method is used in managing inline data compression in storage systems. A request is received to flush data of a file system to a storage device of a storage system. A compression ratio is determined for writing the data to the storage device in a compressed format. Based on the determination, a number of allocation units is determined in which a segment is apportioned. The segment comprises of a set of allocation units in the compressed format. Based on the determination, the data is written to the segment in the compressed format.

Abstract: A storage processor in a data storage system includes a compression selection component that selects a data compression component to be used to compress host I/O data that is flushed from a persistent cache of the storage processor based on a current fullness level of the persistent cache. The compression selection component selects compression components implementing compression algorithms having relatively lower compression ratios for relatively higher current fullness levels of the persistent cache, and selects compression components implementing compression algorithms having relatively higher compression ratios for relatively lower current fullness levels of the persistent cache.

Abstract: A technique for managing data received into a cache operates in cycles. To process a current batch of compressed blocks during a current cache processing cycle, a storage system obtains a new mapping structure and a new segment of contiguous storage space. If the system can place some of the current batch of compressed blocks into previously-allocated space, the system does so and partially populates the new mapping structure with entries for mapping the other compressed blocks that were not placed. The system then asserts a hold on the new mapping structure, so that the mapping structure is retained in cache at the end of the current cache processing cycle, and more completely populates the new mapping structure with entries for other compressed blocks during a later cache processing cycle before releasing the hold.

Abstract: A technique for managing data received into a cache operates in cycles. To process a current batch of compressed blocks during a current cache processing cycle, a storage system obtains a new mapping structure and a new segment of contiguous storage space. If the system can place some of the current batch of compressed blocks into previously-allocated space, the system does so and partially populates the new mapping structure with entries for mapping the other compressed blocks that were not placed. The system then asserts a hold on the new mapping structure, so that the mapping structure is retained in cache at the end of the current cache processing cycle, and more completely populates the new mapping structure with entries for other compressed blocks during a later cache processing cycle before releasing the hold.

Abstract: A technique for writing data in a data storage system includes aggregating data received in a set of I/O requests into a batch that includes multiple extents of data. After compressing a current extent of the batch and determining that the compressed extent does not fit in a space where a previous version of the extent is stored, the technique performs a batch-relocate operation by gathering a set of mapping metadata for mapping each of the extents in the batch, identifying a set of holes indicated by the set of mapping metadata, and adding the holes to a batch-hole list. The technique then selects a hole, from the batch-hole list, which is big enough to accommodate the compressed extent, and places the compressed extent in the selected hole.

Abstract: A technique for writing data in a file system includes aggregating data received in a set of I/O requests into a batch that includes multiple extents of data. After compressing a current extent of the batch and determining that the compressed extent does not fit in a space where a previous version of the extent is stored, the technique performs an FS-relocate operation by accessing an F S-hole list provided for the file system and selecting a hole, from the FS-relocate list, which is large enough to accommodate the compressed extent. The technique then places the compressed extent in the selected hole.

Abstract: A technique for writing data in a file system includes aggregating data received in a set of I/O requests into a batch that includes multiple extents of data. After compressing a current extent of the batch and determining that the compressed extent does not fit in a space where a previous version of the extent is stored, the technique performs an FS-relocate operation by accessing an FS-hole list provided for the file system and selecting a hole, from the FS-relocate list, which is large enough to accommodate the compressed extent. The technique then places the compressed extent in the selected hole.

Abstract: A technique for writing data in a data storage system includes aggregating data received in a set of I/O requests into a batch that includes multiple extents of data. After compressing a current extent of the batch and determining that the compressed extent does not fit in a space where a previous version of the extent is stored, the technique performs a batch-relocate operation by gathering a set of mapping metadata for mapping each of the extents in the batch, identifying a set of holes indicated by the set of mapping metadata, and adding the holes to a batch-hole list. The technique then selects a hole, from the batch-hole list, which is big enough to accommodate the compressed extent, and places the compressed extent in the selected hole.

Abstract: A technique for managing write requests in a data storage system checks whether newly-arriving data match previously-stored data that have been recorded in a deduplication database. If a match is found, the technique compares mapping metadata for the newly-arriving data with mapping metadata for the matching data. If both sets of metadata point to the same storage location, then the newly-arriving data is a same-data write and a new write to disk is avoided.

Abstract: A storage processor in a data storage system includes a compression selection component that selects a data compression component to be used to compress host I/O data that is flushed from a persistent cache of the storage processor based on a current fullness level of the persistent cache. The compression selection component selects compression components implementing compression algorithms having relatively lower compression ratios for relatively higher current fullness levels of the persistent cache, and selects compression components implementing compression algorithms having relatively higher compression ratios for relatively lower current fullness levels of the persistent cache.

Abstract: A technique for overwriting compressed data tests whether new data compressed with a first compression procedure fits within spaces provided for previous data. If the compressed new data does not fit, the technique compresses the new data using a second compression procedure. Assuming the second compression procedure reduces the compressed size of the new data to fit the available space, the technique stores the new data in the same location as the previous data. In this manner, overwrites can be accommodated in place without the need to create new mapping metadata.

Abstract: A method is used in managing data deduplication in storage systems based on storage space characteristics. Characteristics of first and second storage tiers are evaluated. A first data object resides on the first storage tier and a second data object resides on the second storage tier. The first and second data objects are selected for applying a deduplicating technique. A data storage system includes the first and second storage tiers configured such that performance characteristics associated with the first storage tier is different from the second storage tier. Based on the evaluation, the deduplicating technique is applied to the first and second data objects.

Abstract: A method is used in managing data deduplication in storage systems based on I/O activities. I/O activity of first and second data objects is evaluated. The first and second data objects are selected for applying a deduplicating technique. Based on the evaluation, the deduplicating technique is applied to the first and second data objects.

Abstract: A method is used in managing data deduplication in storage systems. A data deduplication process is performed by applying a deduplicating technique to data of a deduplication domain. The data deduplication process is scheduled based on a priority. Characteristics of data deduplication performed by the data deduplication process are evaluated. Based on the evaluation, execution of the data deduplication process is effected.

Abstract: A method is used in managing data deduplication in storage systems. Characteristics of first and second data objects are evaluated based on a set of deduplication policies. The first and second data objects are selected for deduplication. Based on the evaluation, deduplication performed on the first and second data objects is affected.

Abstract: A storage processor of a data storage system includes a data relocation task scheduler that provides system limit control on the execution of data relocation requests. The data relocation task scheduler also provides fairness and concurrency limit enforcement by scheduling relocation requests from different provisioning domains, such as storage pools, in a fair manner. For example, the data relocation task scheduler includes stackable sets of pending task queues that store incoming data relocation requests. The data relocation task scheduler applies an iteration scheme to remove data relocation requests from the pending task queues thereby providing scheduling fairness among the queues and among the different provisioning domains within the data storage system.