Abstract: Described are techniques for performing data movement optimization processing comprising: receiving a list of proposed data movements; ranking the list in accordance with one or more criteria associated with each of the proposed data movements of the list, wherein the one or more criteria for each proposed data movement, that moves a data portion to a target storage tier, includes at least one characteristic about each proposed data movement other than I/O workload directed to the data portion; selecting a subset of the proposed data movements of the list; implementing the subset of the proposed data movements by performing each of the proposed data movements of the subset; and revising the list to remove proposed data movements of the subset implemented in said implementing. A new list may be obtained each time period. A subset of the ranked list may be implemented each time period in a cycle.

Abstract: A method, computer program product, and computer system for identifying, by a computing device, information associated with a relationship between a physical layer block and a virtual logic block for RAID storage. The information associated with the relationship between the physical layer block and the virtual logic block may be written within the RAID storage. The physical layer block within the RAID storage may be rebuilt only when the physical layer block includes the information associated with the relationship between the physical layer block and the virtual logic block.

Abstract: In a hybrid storage array that implements hierarchical storage tiering the eviction of host application data from cache is coordinated with promotion and demotion of host application data between hierarchical storage tiers. Optimal distribution of read cache size per different storage objects may be determined based on cache miss cost. The cost or benefit of promotion and demotion may be determined based on read cache hits and misses.

Abstract: Implementations are provided herein for systems, methods, and a non-transitory computer product configured to analyze an input/output (IO) pattern for a data storage system, to identify an application type based on the IO pattern, and to select optimal deduplication and compression configurations based on the application type. The teachings herein facilitate machine learning of various metrics and the interrelations between these metrics, such as past IO patterns, application types, deduplication configurations, compression configurations, and overall system performance. These metrics and interrelations can be stored in a data lake. In some embodiments, data objects can be segmented in order to optimize configurations with more granularity. In additional embodiments, predictive techniques are used to select deduplication and compression configurations.

Abstract: Techniques for data storage optimization processing that may include: receiving I/O access information characterizing observed I/O operations directed to a logical address space; analyzing the I/O access information to identify a predicted I/O access distribution expected for the plurality of logical regions of the logical address space at multiple points in time; determining a placement plan specifying placement of data portions of the plurality of logical regions among a plurality of storage tiers; and performing, in accordance with the placement plan, at least one data movement that moves at least a first data portion of a first of the plurality of logical regions from a first of the plurality of storage tiers to a second of the plurality of storage tiers.

Abstract: Techniques for processing I/O operations may include: receiving a write I/O operation that writes first data to a target logical address in a logical address range, wherein physical storage is currently unallocated for the target logical address; selecting a first storage tier from which to allocate first physical storage for storing the first data, wherein said selecting uses first I/O access information characterizing I/O activity of a first logical region and performance and capability characteristics of storage tiers to select the first storage tier; allocating first physical storage from the first storage tier; and storing the first data in the first physical storage. The target logical address may be located in the first logical region. The target logical address may alternatively be located in another logical region predicted, based on temporal trends and application profile information, to include the target logical address at a future point in time.

Abstract: Techniques for processing I/O operations may include: receiving, at a data storage system, a write operation that writes first data to a target logical address of a log, wherein the data storage system includes a first storage tier of rotating non-volatile storage devices and a second tier of non-volatile solid state storage devices; storing the first data of the target logical address in a first level cache; destaging the first data from the first level cache to a first physical storage location in the first storage tier; and determining, in accordance with first read activity information for the target logical address, whether to store the first data for the target logical address in a second level cache including at least a portion of the non-volatile solid state storage devices of the second tier. The second level cache is a content addressable caching layer that caches data based on read activity.

Abstract: Implementations are provided herein for systems, methods, and a non-transitory computer product configured to analyze an input/output (IO) pattern for a data storage system, to identify an application type based on the IO pattern, and to select optimal deduplication and compression configurations based on the application type. The teachings herein facilitate machine learning of various metrics and the interrelations between these metrics, such as past IO patterns, application types, deduplication configurations, compression configurations, and overall system performance. These metrics and interrelations can be stored in a data lake. In some embodiments, data objects can be segmented in order to optimize configurations with more granularity. In additional embodiments, predictive techniques are used to select deduplication and compression configurations.

Abstract: Implementations are provided herein for systems, methods, and a non-transitory computer product configured to analyze an input/output (IO) pattern for a data storage system, to identify an application type based on the IO pattern, and to select optimal deduplication and compression configurations based on the application type. The teachings herein facilitate machine learning of various metrics and the interrelations between these metrics, such as past IO patterns, application types, deduplication configurations, compression configurations, and overall system performance. These metrics and interrelations can be stored in a data lake. In some embodiments, data objects can be segmented in order to optimize configurations with more granularity. In additional embodiments, predictive techniques are used to select deduplication and compression configurations.

Abstract: There is disclosed techniques for use managing data storage. In one embodiment, endurance values are generated in connection with a plurality of solid state drives (SSDs). Each endurance value for an SSD indicating an estimated number of write operations that may be performed on the SSD before the SSD wears out and requires replacement. Additionally, storage space is reserved on one or more of the SSDs such that an endurance level associated with the endurance value of the SSD will have an inverse relationship with the amount of storage space reserved on the SSD.

Abstract: Implementations are provided herein for systems, methods, and a non-transitory computer product configured to analyze an input/output (IO) pattern for a data storage system, to identify an application type based on the IO pattern, and to select optimal deduplication and compression configurations based on the application type. The teachings herein facilitate machine learning of various metrics and the interrelations between these metrics, such as past IO patterns, application types, deduplication configurations, compression configurations, and overall system performance. These metrics and interrelations can be stored in a data lake. In some embodiments, data objects can be segmented in order to optimize configurations with more granularity. In additional embodiments, predictive techniques are used to select deduplication and compression configurations when certain regarding an application type is lacking.

Abstract: A method, computer program product, and computing system for managing wear balance in a mapped RAID storage system. According to embodiments, mapped RAID extents, which are comprised of storage disk extents, are assigned to particular mapped RAID groups based on one or more parameters related to wear experienced by disk extents associated with the RAID extent. Endurance parameters are measured and can be used by machine learning modules to predict future wear levels enabling predictive wear balancing in mapped RAID storage systems. Embodiments can be used when initially forming a mapped RAID group, when adding storage to an existing mapped RAID group, or when managing the ongoing performance of a mapped RAID group or storage system.

Abstract: There is disclosed techniques for use in managing data storage in data storage systems. For example, in one embodiment, the techniques comprise monitoring I/O operations directed to a storage object in a data storage system. The techniques also comprise determining a measure of I/O trend relating to the storage object in response to the said monitoring. The techniques further comprise migrating data associated with the storage object from one tier of storage to another tier of storage in the data storage system based on the said measure of I/O trend.

Abstract: There is disclosed herein techniques for use in managing data storage. In one embodiment, the techniques comprise defining, for each of a plurality of data storage drives, one or more areas on a data storage drive such that each area on the data storage drive corresponds to an area associated with similar I/O characteristics on the other data storage drives. The techniques also comprise selecting two or more drive extents from corresponding areas on different data storage drives of the plurality of data storage drives. The techniques further comprise forming a RAID extent based on the selected drive extents.

Abstract: A technique for use in balancing flash drive wear in data storage systems is disclosed. Multiple sets of flash drives are identified where data is stored as multiple slices striped across the set of flash drives. A write rate at which data will be written to the multiple slices stored on the set of flash drives during a next time interval is predicted. A number of bytes that can be written to each set of flash drives is determined. A metric representative of a wear rate is determined for each set of flash drives. HDD relocation candidates are identified and a relocation process to relocate identified slices initiated.

Abstract: A technique for use in balancing flash drive wear in data storage systems is disclosed. multiple sets of flash drives are identified where data is stored as multiple slices striped across the set of flash drives. A write rate at which data will be written to the multiple slices stored on the set of flash drives during a next time interval is predicted. A number of bytes that can be written to each set of flash drives is determined. A wear metric representative of a wear rate is determined for each set of flash drives. A write quota and an interval period is determined for one or more sets of flash drives. If the wear metric exceeds the write quota for one or more sets of flash drives during the time interval, reduce the wear rate for the one or more sets of flash drives.

Abstract: Load balancing in a mapped RAID data storage system that moves slices of logical unit (LUN) address space between non-volatile storage represented by different rotation groups of RAID extent entries in a RAID extent table. The RAID extent table is divided into RAID extent groups corresponding to partnership groups of drives, and the RAID extent groups are divided into rotation groups. At least one LUN with an address space made up of slices is generated for each RAID extent group. Host data directed to each slice is stored in drive extents indicated by the RAID extent entries in the rotation group to which the slice is mapped. A rebalancing operation modifies the mapping between slices in a LUN and the rotation groups in the corresponding RAID extent group such that at least one slice is remapped from a heavily loaded rotation group to a lightly loaded rotation group.

Abstract: A technique for use in balancing flash drive wear in data storage systems is disclosed. multiple sets of flash drives are identified where data is stored as multiple slices striped across the set of flash drives. A write rate at which data will be written to the multiple slices stored on the set of flash drives during a next time interval is predicted. A number of bytes that can be written to each set of flash drives is determined. A metric representative of a wear rate is determined for each set of flash drives. Flash drive wear rate is balanced such that the wear rate for each of the multiple is approximately equal.

Abstract: Load balancing in a mapped RAID data storage system that moves slices of logical unit (LUN) address space between non-volatile storage represented by different rotation groups of RAID extent entries in a RAID extent table. The RAID extent table is divided into RAID extent groups corresponding to partnership groups of drives, and the RAID extent groups are divided into rotation groups. At least one LUN with an address space made up of slices is generated for each RAID extent group. Host data directed to each slice is stored in drive extents indicated by the RAID extent entries in the rotation group to which the slice is mapped. A rebalancing operation modifies the mapping between slices in a LUN and the rotation groups in the corresponding RAID extent group such that at least one slice is remapped from a heavily loaded rotation group to a lightly loaded rotation group.

Abstract: A technique for use in balancing flash drive wear in data storage systems is disclosed. multiple sets of flash drives are identified where data is stored as multiple slices striped across the set of flash drives. A write rate at which data will be written to the multiple slices stored on the set of flash drives during a next time interval is predicted. A number of bytes that can be written to each set of flash drives is determined. A metric representative of a wear rate is determined for each set of flash drives. Flash drive wear rate is balanced such that the wear rate for each of the multiple is approximately equal.