Abstract: Flexibly expanding the storage capacity of a data storage system by adding a single physical storage device or any number of disk drives to an existing storage system without the need to reconfigure existing erasure encoding groups of the system. The physical storage devices of a data storage system may be divided into a plurality of slices, and each slice may be a member of an erasure encoding group. Physical storage devices that are added to the data storage system may be divided into same number of slices and/to slices of a same size, which then may be added to existing erasure encoding groups, utilized as spare slices or left idle until all of the slices are integrated into the data storage system.

Abstract: The system, devices, and methods disclosed herein relate to using historic data storage system utilization metrics to automate data expansion capacity. In some embodiments, the data storage system is a RAID cloud having a plurality of storage devices divided into logical slices, also called splits. Methods disclosed herein allow user defined thresholds to be set wherein the thresholds control when data storage will be expanded. Data storage expansion capacity can be based upon historic usage criteria or customer-based criteria. Additional system customizations include user control over rebalancing data distribution as well as determining when system performance should be routinely evaluated offline.

Abstract: Described are techniques for performing data migration for a source logical volume and a target. The target is configured as storage for another mirror of the source logical volume prior to copying data from the source logical volume to the target, and if the target is configured storage of another logical volume of the data storage system, the configured storage is remapped as storage for another mirror of the source logical volume prior to copying data from the source logical volume to the target. One or more invalid bits are set indicating that the target does not contain a valid copy of data from the source logical volume. Data is copied from the first mirror of the source logical volume to the target. Invalid bits are cleared as data portions of the first mirror of the source logical volume are copied to the target.

Abstract: Described are techniques for processing a data operation in a data storage system. A front-end component of the data storage system receives the data operation. In response to receiving the data operation, the front-end component performs first processing. The first processing includes determining whether the data operation is a read operation requesting to read a data portion which results in a cache miss; and if said determining determines that the data operation is a read operation resulting in a cache miss, performing read miss processing. Read miss processing includes sequential stream recognition processing performed by the front-end component to determine whether the data portion is included in a sequential stream.

Abstract: Scheduling jobs for a plurality of logical devices associated with physical devices includes assigning a physical run count value and a physical skip count value to each of the physical devices, at each iteration, examining the physical skip count value and the physical run count value for each of the physical devices, and scheduling a number of jobs up to the physical run count value for logical devices associated with a particular one of the physical devices at each iteration corresponding to the physical skip count value for the particular one of the physical devices. The physical skip count value and the physical run count value for a particular one of the physical devices may vary according to a total load of the particular physical device.

Abstract: Scheduling jobs for a plurality of devices includes assigning a run count value to each of the devices, scheduling a number of consecutive jobs up to the run count value, and decrementing the run count value according to the number of consecutive jobs scheduled. The run count value for a particular one of the devices may vary according to a total load of the particular one of the devices. The total load of the particular device may vary according to a number of pending jobs for the particular device and a total number of jobs for all devices coupled to a controller for the devices. The devices may be disk drive devices. Only one read job may be scheduled for a device irrespective of the run count value for the device and devices having pending read jobs may be given precedence over other devices. Multiple read jobs may be scheduled for a device according to the run count value and devices having pending read jobs may be given precedence over other devices.

Abstract: Described are techniques for representing and using RAID group consistency information. RAID group consistency information for a slice of data included on a device may be obtained by mapping the slice to an index. The index is one of a plurality of indices each associated with a structure indicating RAID group inconsistencies for a different portion of slices of data of the device. If the index is not associated with a structure, it may be determined that there is no RAID group inconsistency for the slice. If the index is associated with a structure, a hint bit associated with the slice may be determined wherein the hint bit encodes summary consistency information for a plurality of slices. A determination is made as to whether the hint bit is set, and, if the hint bit is not set, it is determined that there is no RAID group inconsistency for the slice.

Abstract: A scheduling method and apparatus for use by a processor that controls storage devices of a data storage system is presented. The method allocates processing time between I/O operations and background operations for predetermined time slots based on an indicator of processor workload.

Abstract: Described are techniques for initializing a device in a data storage system. An indicator is set in an entry associated with a portion of the device indicating that the portion of the device is to be initialized. A signature field of the entry is set with a signature value indicating that the portion of the device is to be initialized. The device is indicated as available for use after setting the indicator and the signature field for each portion of the device. The device has one or more portions and each of the portions has a corresponding entry including an indicator and a signature field.

Abstract: Described are techniques for verifying data. A write operation request for writing to a first data block is received. A first copy of an identifier for said first data block in a memory is updated. A second copy of the identifier for the first data block is updated. The second copy is stored with the first data block on a data storage device. A read operation request is received for the first data block. The first data block and an associated identifier from the data storage device are read. Data verification processing is performed for the first block in accordance with the associated identifier, the first copy from the memory, and any pending write operations for the first data block.

Abstract: Described are techniques for verifying access to data on a device. A scrubbing deadline is received indicating an ending time by which a current scrubbing cycle is to complete. It is determined whether the scrubbing deadline will be met in accordance with one or more scrubbing criteria if device scrubbing operations are performed in a first mode. Device scrubbing operations are performed in the first mode if it is determined that the scrubbing deadline will be met in accordance with the one or more scrubbing criteria, and otherwise, performing device scrubbing operations in a second mode.

Abstract: A scheduling method and apparatus for use by a processor that controls storage devices of a data storage system is presented. The method allocates processing time between I/O operations and background operations for predetermined time slots based on an indicator of processor workload.

Abstract: A scheduler for selecting a logical volume for scheduling pending I/O tasks for job generation. The scheduler includes a main scheduler that selects from among multiple sub-schedulers that operate according to different scheduling approaches, e.g., high priority read I/O request scheduling, urgency of non-read I/O tasks, as well as load balancing and round robin schemes. In selecting a sub-scheduler, the main scheduler considers the amount of time allocated to performing read I/O requests, as well as the level of urgency associated with the logical volumes and the disk adapter that manages those logical volumes for non-read I/O tasks.