Abstract: Technologies are described herein for providing networked RAID in a virtualized storage cluster. The storage capacity of a storage cluster having two or more storage nodes is organized into tiers. A portion of the available storage capacity is allocated to one tier that is organized using chained declustering. Another portion of the available storage capacity is allocated to another tier that is organized using a networked RAID configuration. The storage cluster monitors the frequency at which data in the storage cluster is accessed. Frequently used data that is stored in the network RAID-configured tier is promoted to the chained declustered-configured tier. Infrequently used data that is stored in the chained declustered-configured tier is demoted to the network RAID-configured tier.

Abstract: A method, system, apparatus, and computer-readable medium are provided for performing read-ahead operations for sequential read operations. A method includes maintaining a bitmap including a plurality of bits, each bit corresponding to a sector of the disk cache and containing data indicating whether the corresponding sector is valid and can be used to satisfy read requests. The method includes receiving a request to read a sector of the disk cache, and in response, identifying a bit in the bitmap that corresponds to the requested sector. Further, the method includes determining whether the disk cache contains valid data for a sector previous to the requested sector by examining a bit in the bitmap previous to the bit that corresponds to the requested sector, and in response, reading sequentially into the disk cache sectors of the disk cache corresponding to bits in the bitmap following the bit corresponding to the requested sector.

Abstract: Technologies are described herein for boot caching to accelerate boot access in a data storage system providing boot consolidation. Boot caching may be provided to improve boot access for multiple clients concurrently booting from a common storage server. Boot statistics may be collected per user or client. The boot statistics can determine common patterns within the boot procedures of the clients and may be used to compute boot caching tables. These tables may then be used during boot caching to accelerate booting of clients. A boot caching mechanism can be implemented to support improved booting performance by caching data blocks common to multiple booting volumes. This approach can leverage the condition that much of the data loaded by clients at boot time may be substantially identical, because booting can involve loading the same operating system components into each client.

Abstract: Technologies are described herein for achieving data consistency during a failover from a primary node to a secondary node in a storage cluster with a shared RAID array in a degraded mode. When the primary storage node receives a write I/O operation, a volume module on the primary node reads the missing data strip data from the failed disk belonging to the stripe targeted by the write I/O operation. The primary storage node communicates the missing data strip to the secondary storage node, which writes the missing strip data to an outstanding strip log. Upon the failure of the primary node, the secondary storage node reads the missing data strip from the outstanding strip log and writes the missing data strip to the shared RAID array, thus restoring data consistency to the stripe of the RAID array containing the missing data strip.

Abstract: A method, system, apparatus, and computer-readable medium are provided for implementing caching in a data storage system. According to aspects of the invention, a cache module is provided that utilizes cache lines sized according to a snapshot chunk size or an allocation unit size. The cache module utilizes cache header data structures corresponding to cache lines, each of which is assigned a device and logical block access range when active. The active headers are arranged in a set of hash queues. A free queue corresponds to the list of unused cache headers and a dirty queue corresponding to a list of unflushed cache headers. The cache header contains sector-level bitmaps of the cache line, specifying on a per sector granularity the bits that are dirty and valid. Flushing is performed by copying the dirty bitmap into a temporary memory location and flushing the bits set in it, while resetting the dirty bitmap and allowing writes to it.

Abstract: A method, system, and apparatus for expanding the storage capacity of a data storage system are provided. According to one method, the physical storage devices that store the data and metadata for one or more logical disks are organized into a container. A logical disk segment is created within the container for each logical disk. When storage capacity is added through the addition of logical disks, a new logical disk segment is added to the end of the container for the new logical disk. When storage capacity is added through the addition of capacity to an existing logical disk, a new logical disk segment is added to the end of the container for the new logical disk segment within the logical disk. The devices within a container may be migrated between server computers or between CPUs with their associated mass storage controllers.

Abstract: Technologies are described herein for boot caching to accelerate boot access in a data storage system providing boot consolidation. Boot caching may be provided to improve boot access for multiple clients concurrently booting from a common storage server. Boot statistics may be collected per user or client. The boot statistics can determine common patterns within the boot procedures of the clients and may be used to compute boot caching tables. These tables may then be used during boot caching to accelerate booting of clients. A boot caching mechanism can be implemented to support improved booting performance by caching data blocks common to multiple booting volumes. This approach can leverage the condition that much of the data loaded by clients at boot time may be substantially identical, because booting can involve loading the same operating system components into each client.

Abstract: Technologies are described herein for providing networked RAID in a virtualized storage cluster. The storage capacity of a storage cluster having two or more storage nodes is organized into tiers. A portion of the available storage capacity is allocated to one tier that is organized using chained declustering. Another portion of the available storage capacity is allocated to another tier that is organized using a networked RAID configuration. The storage cluster monitors the frequency at which data in the storage cluster is accessed. Frequently used data that is stored in the network RAID-configured tier is promoted to the chained declustered-configured tier. Infrequently used data that is stored in the chained declustered-configured tier is demoted to the network RAID-configured tier. The zone size for zones stored in the chained declustered-configured tier may be different than for zones stored in the network RAID-configured tier.

Abstract: Technologies are described herein for achieving data consistency during a failover from a primary node to a secondary node in a storage cluster with a shared RAID array in a degraded mode. When the primary storage node receives a write I/O operation, a volume module on the primary node reads the missing data strip data from the failed disk belonging to the stripe targeted by the write I.O operation. The primary storage node communicates the missing data strip to the secondary storage node, which writes the missing strip data to an outstanding strip log. Upon the failure of the primary node, the secondary storage node reads the missing data strip from the outstanding strip log and writes the missing data strip to the shared RAID array, thus restoring data consistency to the stripe of the RAID array containing the missing data strip.

Abstract: Technologies are described herein for high-performance and space efficient journaling mechanisms. A journal queue can buffer incoming I/O operations. The queue can be read by a journal write module. The journal write module can form metadata headers to store into the journal along with each I/O operation. Compound metadata headers may be formed for multiple I/O operations and the combination of multiple I/O operations may be efficiently stored to the journal as one journal write. Multiple journal entries may be written into the journal in parallel while maintaining the sequential ordering of the journal. A queue depth threshold can be established for determining if journal entries should be combined into compound entries or not. Multiple concurrent readers can support continuous data protection, and various data replication features. Each reader can be represented as a reader pointer and a journal writer can invalidate reader pointers when related data is overwritten.

Abstract: A method, system, apparatus, and computer-readable medium are provided for storing data at a virtual tape library (“VTL”) computer or server. According to one method, a VTL computer maintains one or more storage volumes for use by initiators on an array of mass storage devices. Space on each of the volumes is allocated using thin provisioning. The VTL computer may also include a cache memory that is at least the size of a full stripe of the array. Write requests received at the VTL computer are stored in the cache memory until a full stripe of data has been received. Once a full stripe of data has been received, the full stripe of data is written to the array at once. The array utilized by the VTL computer may include a hot spare mass storage device. When a failed mass storage device is identified, only the portions of the failed device that have been previously written are rebuilt onto the hot spare. The array may be maintained using RAID-5.

Abstract: Techniques for continuous data protection can include creating snapshots of one or more underlying storage volumes upon specific file system events. Generating snapshots upon every file close event can protect the files in a storage system by keeping a snapshot of every version or modification of each file. Removal of redundant snapshots can mitigate the impact on storage capacity associated with creating these large numbers of volume snapshots upon each file close event. Additionally, file closure lists can be employed to allow generating snapshots only when a previously closed file is reopened. Such an approach can protect the previous version of a file prior to the opening of a new version of the file. Such an approach can also mitigate storage capacity impact without the creation of redundant snapshots.

Abstract: Technologies are described herein for providing networked RAID in a virtualized storage cluster. The storage capacity of a storage cluster having two or more storage nodes is organized into tiers. A portion of the available storage capacity is allocated to one tier that is organized using chained declustering. Another portion of the available storage capacity is allocated to another tier that is organized using a networked RAID configuration. The storage cluster monitors the frequency at which data in the storage cluster is accessed. Frequently used data that is stored in the network RAID-configured tier is promoted to the chained declustered-configured tier. Infrequently used data that is stored in the chained declustered-configured tier is demoted to the network RAID-configured tier. The zone size for zones stored in the chained declustered-configured tier may be different than for zones stored in the network RAID-configured tier.

Abstract: Technologies are provided herein for asynchronous data replication. A primary server maintains and exposes a storage volume for use by storage clients. The primary server receives write operations directed toward the storage volume and performs the writes on the storage volume. The primary server also creates buckets containing each of the write operations. The buckets are opened at a first point in time and closed at a second point in time that may be selected based upon the contents of the buckets, a predefined time period, or based upon an indication from an application program. The buckets are transmitted to a second server computer asynchronously. When each bucket is received completely by the second server, its contents are written to a storage volume maintained by the second computer. The buckets may be created from pages of a write-back cache utilizing a zero buffer copy mechanism.

Abstract: A method, system, and apparatus for expanding the storage capacity of a data storage system are provided. According to one method, the physical storage devices that store the data and metadata for one or more logical disks are organized into a container. A logical disk segment is created within the container for each logical disk. When storage capacity is added through the addition of logical disks, a new logical disk segment is added to the end of the container for the new logical disk. When storage capacity is added through the addition of capacity to an existing logical disk, a new logical disk segment is added to the end of the container for the new logical disk segment within the logical disk. The devices within a container may be migrated between server computers or between CPUs with their associated mass storage controllers.