Abstract: Described are techniques for partitioning resources. A plurality of resource limit ranges are specified for a plurality of tenants of a system. Each of the plurality of resource limit ranges have a lower bound and an upper bound. A plurality of current resource limits are determined. Each of the plurality of current resource limits indicate a current resource limit for one of the plurality of tenants. Each current resource limit for one of the tenants is a value included in one of the plurality of resource limit ranges specified for the one tenant. The plurality of current resource limits are dynamically determined in accordance with a current state of the system.

Abstract: Described are techniques for processing data operations. A read request for first data is received at a data storage system. It is determined whether the read request results in a cache hit whereby the first data is stored in a cache of the data storage system, or whether the read request otherwise results in a cache miss. If the read request results in a cache miss, processing is performed to determine determining whether to perform cacheless read processing or deferred caching processing to service the read request. Determining whether to perform cacheless read processing or deferred caching processing is performed in accordance with criteria including a measurement indicating a level of busyness of a back-end component used to retrieve from physical storage any portion of the first data not currently stored in the cache.

Abstract: Techniques for combined memory and storage tiering. For example, in one example, a method for managing placement of data in a data memory and storage system environment including at least one host computing device and at least one storage array includes the following steps. A memory and storage tier architecture is maintained across the data memory and storage system environment including one or more tiers resident on the host computing device and one or more tiers resident on the storage array. A user is enabled to: (i) specify on which of the one or more tiers resident on the host computing device and the one or more tiers resident on the storage array to store data associated with the execution of an application program; and/or (ii) specify a level of service by which the system automatically manages which of the one or more tiers resident on the host computing device and the one or more tiers resident on the storage array to store data associated with the execution of the application program.

Abstract: Described are techniques for evaluating data movement alternative. A set of criteria including capacity and performance limits is received. First processing is performed to evaluate a plurality of alternatives for use in data movement with respect to a set of logical devices having data stored on a set of physical storage devices. Each of the plurality of alternatives includes a different set of data movement criteria comprising capacity limits and a different set of performance limits. The set of physical storage devices includes at least a first physical device of one of a plurality of storage tiers and a second physical device of another one of the plurality of storage tier. One of the sets of performance limits is selected in accordance with the first processing.

Abstract: Techniques are described for processing a data operation that writes first data to a data portion currently stored on first physical storage of a first storage tier of a data storage system, including storage tiers each having a different performance ranking relative to other storage tiers. Second physical storage is allocated from one of the storage tiers for storing the first data written to the data portion. This one storage tier is selected independent of the first storage tier. The first data is stored in a cache and destaged from the cache to the second physical storage. A single merged copy of data for the data portion is formed. Based on an activity level of the data portion, a target tier is selected for storing the single merged copy of data for the data portion.

Abstract: Described are techniques for partitioning storage of a data storage system. A plurality of storage groups is selected. Each of the plurality of storage groups includes one or more devices and is associated with an allocation policy including a first threshold specifying one of a minimum or a maximum amount of storage of a first of a plurality of storage tiers. Workload data for the plurality of storage groups is analyzed. Storage of a plurality of storage tiers is partitioned for use by the plurality of storage groups in accordance with criteria. The criteria includes workloads of the plurality of storage groups and the allocation policy associated with each of the plurality of storage groups.

Abstract: Techniques are described for determining data movements. Gradient sets are determined for storage tiers. Each gradient set is associated with one of the storage tiers and includes at least one gradient. Based on the gradient sets, a donor tier and a first set of one or more receiving tiers are selected from the plurality of storage tiers. A first data portion having data currently stored on at least one physical device of the donor tier is selected. In accordance with data movement criteria, a second data portion included in a receiving tier of the first set is selected. Selecting the second data portion includes modeling a first data movement including moving the first data portion from the donor tier to the receiving tier. The first data movement may be included in a recommended list and may be performed by a data storage optimizer.

Abstract: Providing quality of service levels to a plurality of sources that perform access requests to a disk resource includes providing a disk resource queue containing access requests for the disk resource, providing a source queue for each of the sources containing access requests generated by a corresponding one of the sources, determining if a new access request from a particular source is urgent according to a specified number of I/O operations per second for the particular source and a time since a previous access request from the particular source, adding the new access request to the disk resource queue if the new access request is urgent, and adding the new access request the source queue of the particular source if the new access request is not urgent and the length of the disk resource queue is greater than a predetermined queue depth value.

Abstract: Described are techniques for performing data storage optimizations. A data storage optimizer on a first data storage system connected to a set of other data storage systems receives performance classification information. The first data storage system includes a first set of storage devices of a first performance classification. The set of other data storage systems includes a second set of storage devices of a second performance classification different from the first performance classification. A write operation is received from the host to write first data to a target address of a virtually provisioned device. It is determined that physical storage is not currently allocated for the target address of the virtually provisioned device. A first portion of physical storage is allocated for storing the first data. The first portion of physical storage is included in a storage device of the second set having the second performance classification.

Abstract: The system described herein allows for picking data to be moved from one physical location to another using certain criteria, such as expected activity level, to achieve load balancing or load optimization. Migrating data for a logical device includes subdividing the logical device into subsections, where each of the subsections corresponds to a particular address range of the logical device that is mapped to a physical device containing corresponding data and copying data for a first subsection from a first physical device to a second, different, physical device, where data for a second subsection is maintained indefinitely on the first physical device after copying the data for the first subsection to the second physical device. The first physical device may be provided in a different geographic location than the second physical device.

Abstract: Described are techniques for discovering physical device dependencies. A set of baseline response times for logical devices of a data storage system are received. A first of the logical devices is selected and a remaining set of logical devices includes each of the logical devices other than the first logical device. Data operations are issued to the first logical device at a first rate while also issuing data operations to the remaining logical devices. A set of observed response times for the remaining logical devices is received. For each remaining logical device, it is determined whether the logical device has a corresponding observed response time that is significantly more the logical device's baseline response time. A first logical device set is determined having physical device dependencies that includes the first logical device set and each remaining logical device having an observed response time significantly greater than it's baseline response time.

Abstract: Described are techniques for automated compression and/or decompression of data. Idle counters are received for data portions which indicate a level of inactivity with respect to idle criteria for the data portions. Compression criteria is received specifying conditions for determining whether a data portion is a candidate for data compression. The compression criteria includes a time compression threshold indicating a minimum amount of time that a data portion has been deemed idle with respect to said idle criteria. A data portion that qualifies as a candidate for data compression has an associated idle counter indicating an inactivity level that is greater than the time compression threshold. Processing determines whether a first data portion is a candidate for data compression in accordance with the compression criteria. If the first data portion is determined as a candidate for data compression, compression processing is performed for the first data portion.

Abstract: Described are techniques for tracking activity for one or more devices. Each of the one or more devices is partitioned into one or more extents. Each of the one or more extents includes a plurality of sub extents. Each of the sub extents includes a plurality of chunks. For each of said one or more extents of each of said one or more devices, extent activity level information is determined including one or more metrics indicating an activity level for said each extent; and For each of said one or more extents of each of said one or more devices, an activity bitmap is determined. The activity bitmap includes an entry for each sub extent included in said each extent, said entry indicating an activity level for said each sub extent.

Abstract: Described are techniques for performing storage optimizations in a system. The optimizations may include a data movement optimization. First processing may be performed by a first component to determine whether to automatically perform the data movement optimization for devices included in one or more data storage systems. The first component may be external with respect to the data storage systems and may not be included in any of the data storage systems. As a result of performing the data movement optimization, a first portion of data is automatically moved from a first of the devices to a second of the devices. The first and second devices may have different performance classifications. Additionally, automated discovery processing may be performed to determine performance classifications for devices of the data storage systems and/or determine physical device dependencies.

Abstract: Described are techniques for caching. At a first point in time, a first set of data portions currently stored in a first cache of a first data storage system is determined. Each data portion of the first set is located on a first device of the first data storage system. Sent to a second data storage system is first information including information identifying a storage location on the first device for each data portion of the first set. The second data storage system includes a second device that is maintained as a mirror of the first device. The storage location for each data portion of the first set is used to identify a second storage location of the second device corresponding to the storage location. The first information is used to populate a second cache of the second data storage system.

Abstract: A method, system and program product for dynamic management of system resources in a data storage system comprising producing a hint at a hint producer; wherein the hint contains information about the system, consuming the hint at a hint consumer, and determining, at the hint consumer, whether or not to act on the information in the hint.

Abstract: Described are techniques for tracking activity for one or more devices. Each of the one or more devices is partitioned into one or more extents. Each of the one or more extents includes a plurality of sub extents. Each of the sub extents includes a plurality of chunks. For each of said one or more extents of each of said one or more devices, extent activity level information is determined including one or more metrics indicating an activity level for said each extent; and For each of said one or more extents of each of said one or more devices, an activity bitmap is determined. The activity bitmap includes an entry for each sub extent included in said each extent, said entry indicating an activity level for said each sub extent.

Abstract: Synchronization of data layouts and resource utilizations at one or more remote replica sites with the workload and data tiering decisions being made at the primary site allows for an efficient and effective workload support transfer in the event of site failover from a primary site to a remote site. Relevant data access information about workload being supported at the primary site is collected and from that raw information, characterized data access information is generated that condenses the raw data access information or otherwise provides relevant encapsulated information about the raw data access information. The characterized data access information is transmitted to the one or more remote sites allowing each remote site to make its own independent decisions on how best to utilize its available resources to match the performance requirements currently being supported by the primary site.

Abstract: Storage requirements of one or more applications may be serviced by a mix of different disk technologies across one or more storage tiers. According to the system described herein, desirable or optimal mixes of the different disks may be determined and used in the efficient and cost-effective modification or upgrade of storage arrays. For example, whereas SATA drives may be the most cost effective per gigabyte of storage, some applications may not be able to operate based on SATA performance characteristics. On the other hand, flash drives, which may have high performance characteristics, may be too expensive to be used exclusively in a storage array. The system described herein provides a tool that allows for optimal planning of storage resources in a multi-tier storage environment responsive to workload data of a storage array and based on performance characteristics and costs of the storage resources.

Abstract: A method, system and program product for dynamic management of system resources in a data storage system comprising producing a hint at a hint producer; wherein the hint contains information about the system, consuming the hint at a hint consumer, and determining, at the hint consumer, whether or not to act on the information in the hint.