Abstract: Technologies are described for implementing a migration mechanism in a storage system containing multiple tiers of storage with each tier having different cost and performance parameters. Access statistics can be collected for each territory, or storage entity, within the storage system. Data that is accessed more frequently can be migrated toward higher performance storage tiers while data that is accessed less frequently can be migrated towards lower performance storage tiers. The placement of data may be governed first by the promotion of territories with higher access frequency to higher tiers. Secondly, data migration may be governed by demoting territories to lower tiers to create room for the promotion of more eligible territories from the next lower tier. In instances where space is not available on the next lower tier, further demotion may take place to an even lower tier in order to make space for the first demotion.

Abstract: Technologies are described for implementing a default migration mechanism in a storage system containing multiple tiers of storage with each tier having different cost and performance parameters. Data that is accessed more frequently can be migrated toward higher performance storage tiers while data that is accessed less frequently can be migrated towards lower performance storage tiers. Default migration behavior can be overridden by user-specified values for tier age residency policy, demotion policy, tier occupancy of volumes, and tier assignment. Data migration can be paused by the user and resumed by the user.

Abstract: Technologies are described for implementing a default migration mechanism in a storage system containing multiple tiers of storage with each tier having different cost and performance parameters. Data that is accessed more frequently can be migrated toward higher performance storage tiers while data that is accessed less frequently can be migrated towards lower performance storage tiers. Default migration behavior can be overridden by user-specified values for tier age residency policy, demotion policy, tier occupancy of volumes, and tier assignment. Data migration can be paused by the user and resumed by the user.

Abstract: Technologies are described for implementing a default migration mechanism in a storage system containing multiple tiers of storage with each tier having different cost and performance parameters. Data that is accessed more frequently can be migrated toward higher performance storage tiers while data that is accessed less frequently can be migrated towards lower performance storage tiers. Default migration behavior can be overridden by user-specified values for tier age residency policy, demotion policy, tier occupancy of volumes, and tier assignment. Data migration can be paused by the user and resumed by the user.

Abstract: Technologies are described for implementing a migration mechanism in a data storage system containing multiple tiers of storage with each tier having different cost and performance parameters. Access statistics can be collected for each territory, or storage entity, within the storage system. Data that is accessed more frequently can be migrated toward higher performance storage tiers while data that is accessed less frequently can be migrated towards lower performance storage tiers. Each tier can be associated with a range of ILM statistics referred to as the bucket for that tier. A pivot table may be provided that relates the tiers and the buckets. Operations on the pivot table can provide counts of how many territories may be promoted or demoted between any two pairs of tiers.

Abstract: Technologies for eliminating duplicate data provisions within a storage system supporting boot consolidation can efficiently identify duplicate data provisions within a data storage system and eliminate duplication by remapping duplicate provisions to point to the same physical storage space. Signatures of provisions within a storage system may be calculated and compared. Matching, or collisions, within the list of provision signatures can indicate candidate provisions for de-duplication. De-duplication territories may be provided as an indirect mapping mechanism in support of the remapping of duplicated provisions. Access statistics associated with provisions within a storage system may be collected. Access statistics can support the scheduling of de-duplication processes. Data de-duplication can support substantial storage space consolidation and significantly improve caching efficiency within a data storage system.

Abstract: Technologies for eliminating duplicate data within a storage system can efficiently identify and eliminate duplication by remapping borrower regions to share physical storage space with lender regions. Block-level de-duplication can co-exist with storage architectures for thin provisioning and snapshot management. Lending maps can track redirected pointers from borrower regions to shared physical storage from lender regions. The lending maps can track the freed status of regions to support efficient write I/O operations without defaulting to unnecessary read-modify-write cycles to complete data writes. Redundancy of de-duplicated data can maintain one or more copies to support recovery from media errors. Candidate regions for de-duplication can be identified by monitoring the times and patterns of data access operations. A sampled mechanism for calculating and comparing signatures of data blocks can support the efficient identification of duplicated data within the storage system.

Abstract: Technologies are described for implementing a migration mechanism in a storage system containing multiple tiers of storage with each tier having different cost and performance parameters. Access statistics can be collected for each territory, or storage entity, within the storage system. Data that is accessed more frequently can be migrated toward higher performance storage tiers while data that is accessed less frequently can be migrated towards lower performance storage tiers. The placement of data may be governed first by the promotion of territories with higher access frequency to higher tiers. Secondly, data migration may be governed by demoting territories to lower tiers to create room for the promotion of more eligible territories from the next lower tier. In instances where space is not available on the next lower tier, further demotion may take place to an even lower tier in order to make space for the first demotion.

Abstract: Technologies are described for implementing container space management (CSM) in a data storage system. Performance may be improved by ensuring that the maximum number of drive spindles is made available to I/O loads. This may be accomplished by distributing data equally amongst all drive spindles, even if the drive spindles are in different logical drives. A workflow analysis may be performed to determine when the I/O load on the storage system is low and thus identify preferred times for background data re-striping. Data may be re-striped in a minimum number of movements using a data placement algorithm and data placement maps to select data to be re-striped. The data may be re-striped such that the most frequently accessed data within a tier is moved first. Such re-striping can be performed in a manner which optimizes the performance of the storage system both during, and after, the re-striping process.

Abstract: Technologies are described for implementing a migration mechanism in a data storage system containing multiple tiers of storage with each tier having different cost and performance parameters. Access statistics can be collected for each territory, or storage entity, within the storage system. Data that is accessed more frequently can be migrated toward higher performance storage tiers while data that is accessed less frequently can be migrated towards lower performance storage tiers. Each tier can be associated with a range of ILM statistics referred to as the bucket for that tier. A pivot table may be provided that relates the tiers and the buckets. Operations on the pivot table can provide counts of how many territories may be promoted or demoted between any two pairs of tiers.

Abstract: Technologies are described for implementing a system that schedules background workflows in a storage system. Historical load statistics can be analyzed to estimate the preferred times in the future during which to schedule background workflows. After collecting load statistics related to the storage system, natural cyclical patterns in the load data can be determined as the predominant periodicities of the load data are selected from peaks in an autocorrelation of the collected data. A linear predictor may be formed as a weighted sum of the historical data sampled at the determined periodicities. Error minimization, such as the method of least squares, may be used to determine the values of the weighting in order to minimize an error between the load predictor and the measured load. Predicting future loads can improve the scheduling of background operations by avoiding conflicts with normal system operations.

Abstract: Technologies are described for implementing a system that schedules background workflows in a storage system. The background workflows may be automated maintenance tasks that require data movement, or any other tasks that would otherwise compete with normal input/output operations. Historical load statistics can be analyzed to estimate the preferred times in the future during which to schedule background workflows. After collecting data access statistics and usage patterns for several days or weeks, a number of averages can be calculated from the statistics at various natural periods of system utilization. For example, daily, weekly, or other periods. A weighted average can be established where weighting can place more emphasis on recent data. Such averages can serve as predictors of future loads having similar periodic relationships to the historical load data. Predicting future loads can improve the scheduling of background operations by avoiding conflicts with normal system operations.

Abstract: Technologies are described for implementing a migration mechanism in a data storage system containing multiple tiers of storage with each tier having different cost and performance parameters. Access statistics can be collected for each territory, or storage entity, within the storage system. Data that is accessed more frequently can be migrated toward higher performance storage tiers while data that is accessed less frequently can be migrated towards lower performance storage tiers. Each tier can be associated with a range of ILM statistics referred to as the bucket for that tier. A pivot table may be provided that relates the tiers and the buckets. Operations on the pivot table can provide counts of how many territories may be promoted or demoted between any two pairs of tiers.

Abstract: Technologies are described for implementing locks to control I/O operations in a storage system. The lock implementations can have reduced resource requirements for memory and processing. Through the utilization of the technologies and concepts presented herein, an I/O process preparing to acquire a write lock can advertise its need for the lock for a pre-determined time before the lock is granted. The time period for advertisement can be specified so that all I/O operations that were initiated before the advertisement are guaranteed to have completed by the time the advertisement period is over. As such, a lock controller may only need to track outstanding I/Os that begin once the advertisement starts. This can reduce the typical requirement to maintain and process a lock structure for every unit of storage in the system at all times.