Abstract: Determining an expected compression rate for a prospective process in a federated system includes obtaining compression rate data for existing processes in the federated system, compiling the compression rate data into a plurality of entries in a process name table according to process identifier, client, and industry, determining a specific entry in the process name table for an existing process that most closely matches the prospective process, and determining an expected compression rate of the prospective process based on the compression rate data for the specific entry. Compression rate data may be provided by a driver at host systems that sends compression rate information to a central repository. The central repository may be provided by a host system at a data center of the federated system. The compression rate data may use a sliding average that weighs the data more heavily to favor more recent data.

Abstract: Compressing data of a storage device includes selecting a portion of data for data compression that is predicted to be unlikely to be accessed, selecting a particular one of a plurality of data compression algorithms to apply to the portion according to a frequency value associated with each of the data compression algorithms, and adjusting the frequency value of the particular one of a plurality of data compression algorithms according to performance of the particular one of a plurality of data compression algorithms. The performance may vary according to a ratio of amount of compression achieved to processing cycles of a processor used to compress the portion of data. The processor may perform storage device functions that are separate from data compression. The portion of data may be selected from a plurality of logical devices used in connection with a single application that accesses the storage device.

Abstract: Execution of I/O operations for an application is managed to improve consistency of performance for an I/O performance metric for the application. Such control may be achieved by defining a value for a performance resilience objective (PRO) parameter, and applying the PRO value to I/O requests of the application. The PRO value may define a value indicative of an extent to which I/O performance can deviate from a performance objective (e.g., as defined by a service level) over time; i.e., a tolerance for such deviation. Components of a computer network may be configured to apply PRO values to I/O requests for one or more applications. The PRO values of applications may be distributed to one or more components of a data storage system and one or more components of a host system, and these components may be configured to apply the PRO values to I/O requests of the subject applications.

Abstract: Determining storage of particular data in cache memory of a storage device includes using a first mechanism to determine when to remove the particular data from the cache memory and using a second mechanism, independent from the first mechanism, to inhibit the particular data from being stored in the cache memory independent of whether the first mechanism otherwise causes the particular data to be stored in the cache memory. The first mechanism may remove data from the cache memory that was least recently accessed. The second mechanism may be based, at least in part, on a prediction value of an expected benefit of storing the particular data in the cache memory. The prediction value may be determined based on input data corresponding to measured cache read hits (RH), cache write hits (WH), cache read misses (RM), cache write destage operations (WD), and prefetch reads (PR) for the particular data.

Abstract: A method of allocating storage volumes between compressed and uncompressed storage tiers includes maintaining a respective state machine for each storage volume, each state machine maintaining a current state of the storage volume, a previous state of the storage volume, and a state machine timer based on when the respective storage volume last changed state. The method further includes allocating a first subset of the storage volumes to an uncompressed storage tier and allocating a second subset of storage volumes to a compressed storage tier, and determining storage volumes to be moved between the uncompressed and compressed storage tiers using a state machine evaluation process based on the state information of the state machines.

Abstract: A technique allocates storage from a plurality of storage pools of a data storage assembly to host storage. The technique involves receiving an allocation request and, in response to the allocation request, performing a set of probability-based allocation attempt operations that attempts to allocate a storage portion from one of the plurality of storage pools to the host storage. Each storage pool of the plurality of storage pools provides multiple storage portions of physical storage to store host data. The technique further involves, based on a result of the set of probability-based allocation attempt operations, provisioning the host storage with a storage portion from a particular pool of the plurality of storage pools to satisfy the allocation request.

Abstract: Techniques are described determining data movements. A first plurality of performance goals for a plurality of storage pools are received. Each of the first plurality of performance goals specifies a performance goal for one of the plurality of storage pools. A second plurality of performance goals for a plurality of applications are received. Each of the second plurality of performance goals specifies a performance goal with respect to I/O operations directed to one or more logical devices used by one of the plurality of applications. A set of proposed data movements between a first of the plurality of storage pools and a second of the plurality of storage pools is determined in accordance with criteria including any of the first plurality of performance goals and the second plurality of performance goals.

Abstract: Described are techniques for controlling I/O response time. It is determined whether a first observed I/O response time for a storage group exceeds I/O performance specified by a service level objective. If so, first processing may be performed to decrease I/O performance for the storage group. The first processing may include: for each I/O directed to the storage group that is received within a first subsequent time period, determining a first response time denoting an amount of time taken to service each I/O prior to returning a response for each I/O; determining whether the first response time for each I/O is less than a response time delay threshold; and if the first response time for each I/O is less than the response time delay threshold, applying a response time delay to each I/O prior to returning a response for each I/O.

Abstract: Techniques are described determining data movements. A first plurality of performance goals for a plurality of storage pools are received. Each of the first plurality of performance goals specifies a performance goal for one of the plurality of storage pools. A second plurality of performance goals for a plurality of applications are received. Each of the second plurality of performance goals specifies a performance goal with respect to I/O operations directed to one or more logical devices used by one of the plurality of applications. A set of proposed data movements between a first of the plurality of storage pools and a second of the plurality of storage pools is determined in accordance with criteria including any of the first plurality of performance goals and the second plurality of performance goals.

Abstract: Techniques are described for performing data storage optimization. A first I/O workload for a first data portion of a first snapshot of a first logical device is tracked. First processing is performed by a data storage optimizer to determine a set of one or more data movement optimizations. The first processing uses the first I/O workload for the first snapshot. The set of one or more data movement optimizations include a first data movement that is any of a promotion to move data included in the first data portion from a first storage tier to a higher performance storage tier and a demotion to move data included in the first data portion from the first storage tier to a lower performance storage tier. The first data movement is performed.

Abstract: Techniques for obtaining consistent read performance are disclosed that may include: receiving measured read I/O (input/output) response times for flash storage devices; and determining, in accordance with a specified allowable variation, whether a first of the measured read I/O response times for a first of the flash storage devices is inconsistent with respect to other ones of the measured read I/O response times. Responsive to determining the first measured read I/O response time is inconsistent first processing may be performed that corrects or alleviates the inconsistency of the first measured read I/O response time. The first processing may include varying the first measured read I/O response time of the first flash storage device by enforcing, for the first flash storage device, a write I/O workload limit a read I/O workload limit and an idle capacity limit. Data portions may be ranked and selected for data movement based on read workload, write workload or idle capacity.

Abstract: A service level is assigned to each application that uses virtual memory. The service level is used to select a type of memory used when paging-in data. The service level is used to select a type of storage used when paging out data. The service level is used to select a page to evict from memory, e.g. based on service level probabilities. The service level is used to select a number of contiguous pages to page-in, e.g. based on a service level scalar. Accesses (hits) to the pages in memory may be tracked, including contiguous pages that are paged-in based on the scalar. Pages with low hit frequency may be evicted. The scalar for an application may be adjusted when at least some of the contiguous pages are infrequently accessed.

Abstract: A virtualized storage array provides deduplication for a primary storage array that supports VVOLs. The VVOLs may be created for VMs that support host applications and guest OSs in a VDI environment. A VVOL is initially created as a virtualized managed drive that is backed by the virtualized storage array. The data is deduped by the virtualized storage array. After deduplication the data is either maintained on the virtualized storage array or moved to the primary storage array based on satisfying deduplication criteria such as reduction ratio.

Abstract: Described are techniques for storing data. A write operation is received that is directed to a target logical address of a logical device where physical storage has not been allocated for the target logical address. The logical device has a service level objective denoting a performance goal to be maintained with respect to I/O operations for the logical device. A storage tier is selected from a plurality of storage tiers from which to allocate storage for storing write data of the write operation. The selecting is performed in accordance with determining a match between an expected level of performance of the storage tier and the performance goal. First physical storage is allocated from the storage tier. A logical address subrange of the logical device is mapped to the first physical storage. The write data is stored in the first physical storage at a location mapped to the target logical address.

Abstract: Data storage optimization techniques determine predicted values for I/O statistics using an ARIMA (auto-regressive integrated moving average) model. The ARIMA model may be used to capture periodic patterns and trends of workload I/O access to predict the future load demand. A current set of I/O statistics is collected for a current time period T. Using the current set and one or more ARIMA models, a predicted set of I/O statistics is determined for a next time period T+1. Each of the ARIMA models is characterized by model parameters including P denoting a number of auto-regressive terms, D denoting a number of nonseasonal difference needed for stationarity, and Q denoting a number of lagged forecast errors of prediction. A data storage optimizer may determine one or more data portions for movement from a current storage tier to a target storage tier using the predicted set of I/O statistics.

Abstract: A method, computer program product, and computing system for determining a load value for each of a plurality of storage targets included within a RAID group, thus defining a plurality of load values. The plurality of load values are compared to determine if the RAID group is load balanced. If the RAID group is not load balanced, data is repositioned between the plurality of storage targets.

Abstract: An external storage resource pool associated with federated tiered storage is associated with at least one performance tier based on evaluated performance of the external storage resource pool. Performance of the external pool may be evaluated in terms of service level expectations or service level objectives. Workload pattern analysis or performance probability curves may be used to classify the external storage resource pool. Workload distribution may be determined by a margin analysis algorithm that matches workloads and storage resource pool performance characteristics.

Abstract: Determining whether to compress data of a virtual storage unit based at least in part on: an I/O activity value of a virtual storage unit; a compressibility value of the virtual storage unit; and/or a capacity utilization of the storage system or a component thereof. For example, decision logic may be configured based on one or more of such parameters such that virtual storage units with relatively high I/O activities are rarely or never compressed, e.g., to avoid the disproportionately high increases in CPU and bandwidth resource consumption and I/O latency this could create. Decision logic may be configured such that, in general, the likelihood that a virtual storage unit will be compressed increases as: the I/O activity of the virtual storage unit decreases; the system capacity utilization increases; and/or as the compressibility value of the virtual storage unit increases.

Abstract: High performance storage resources are proactively deallocated when a storage group exceeds its service level objective. A load score is calculated for each unit of storage allocated from a high performance physical disk group. A minimum load score is determined that is the lowest load score calculated for the storage units allocated from the high performance disk group to the storage group. When the storage group service level objective is exceeded, a demotion threshold is calculated equal to a product of the minimum load score and a demotion range factor. A demotion set is determined made up of storage units allocated from the high performance disk group having load scores less than or equal to the demotion threshold. Host data stored on the demotion set is migrated to storage from a lower performance disk group, and the high performance units of storage in the demotion set are released for re-allocation.

Abstract: Described are techniques for processing service level objectives. A first service level objective specified for a storage group of devices may include a first value denoting a first target level of performance for I/O operations. A second service level objective specified for a first portion of the storage group may include a second value denoting a second target level of performance for I/O operations directed to the first portion. The second value may denote a higher level of performance than the first value. It may be determined whether there is a violation of any of the first service level objective and the second level objective. Responsive to determining there is the violation of any of the first service level objective and the second level objective, one or more data movements in accordance with the violation may be performed.