Abstract: Systems and methods for an incast mitigation approach that first modifies network responses for content requests above a threshold size to be delayed according a response latency selected from an established latency range. Additionally, as incast characteristics are analyzed from network traffic, the volumes can selectively modify the individual established latency range to increase the latency range during periods of higher incast characteristics and to decrease the latency range when incast characteristics appear to be decreasing.

Abstract: A computer-implemented method for predicting a risk of disk failure is described herein. The method can include collecting a performance parameter for respective disks in a group of disks, and calculating an expected value of the performance parameter for the group of disks. The method can also include comparing a respective value of the performance parameter for a respective disk in the group of disks to the expected value, and predicting a risk of disk failure based on the comparison. For example, when a difference between the respective value of the performance parameter for the respective disk and the expected value exceeds a threshold, the respective disk may be at risk of failing.

Abstract: An example computer-implemented method for restoring data consistency in a RAID array can include detecting failure of a disk of the RAID array, maintaining tracking information for write input/output (“I/O”) operations that effect a data block or a parity block stored on the failed disk, re-commissioning the failed disk and re-synchronizing the RAID array using the tracking information. The tracking information can be used to restore consistency to at least one of the data stripes. For example, the RAID array can store data in one or more data stripes, where each data stripe has a plurality of data and parity blocks. Additionally, the data stripes to which consistency is restored can include a data block or a parity block that is stored on the re-commissioned disk and that is effected by the write I/O operations and stored on the re-commissioned disk.

Abstract: Methods and systems for identifying candidates for de-duplication are disclosed herein. An example system may include: a de-duplication lister that coordinates with a kernel module to identify a unit of data for de-duplication and calculates a signature of the unit of data; a de-duplication queue that manages a de-duplication queue; a de-duplication tracker that maintains a de-duplication data structure including a plurality of entries corresponding to units of data and searches the de-duplication data structure to determine whether the signature of the identified unit of data matches the signature of at least one of the entries in the de-duplication data structure; a de-duplication maintainer that reviews the entries of the de-duplication data structure and removes obsolete entries from the de-duplication data structure; and a de-duplication engine that coordinates with the kernel module to perform de-duplication operations on the de-duplication queue.

Abstract: A method for performing cache flushing operations in a data storage system can include maintaining a plurality of SSDs as a cache medium for a data storage medium, controlling a region of the SSDs in a write-back cache mode, and monitoring a status of the SSDs to detect a low-performance condition. In the write-back cache mode, data is mirrored across the SSDs. The method can also include performing normal purge operations on the data stored in the region under a condition that the low-performance condition is not detected, and performing aggressive purge operations on the data stored in the region in response to detecting the low-performance condition. The normal purge operations can include flushing the data stored in the region to the data storage medium. The aggressive purge operations can include sequentially mirroring the data stored in the region to one or more special territories of the data storage medium.

Abstract: Technologies for eliminating duplicate data within a storage system. De-duplication may be performed done at physical chunk level, where the data is not copied or moved to different location. A logical mapping is modified using a thin de-duplication kernel module that resides between a distributed volume manager (DVM) and a logical disk (LD). De-duplication is achieved by changing pointers in the mapping to land at a physical location. De-duplication is performed as post-process feature where duplicates are identified and the duplicates are marked in the mapping table, thereby claiming free space through de-duplication. Block-level de-duplication in accordance with the above can co-exist with existing storage architectures for thin provisioning and snapshot management.

Abstract: Technologies for providing recovery point review within a continuous data protection system can support a data storage operator in a data recovery procedure. A snapshot image can be chosen that was made prior to a data loss event. A new writable snapshot can be created from this logged snapshot. The new writeable snapshot can become the target for recovering the data volume. Review can be performed in a forward direction by reading a journal of I/O operations that have occurred since the prior snapshot was made. Review can be performed in a backward direction by deriving a dirty data list of changes made to the data volume and then removing the changes as requested. The operator can continue forward and backward review operations to refine the selected recovery point. Upon establishing the desired recovery point, the new writable snapshot can be committed, or rolled back, onto the data storage volume.

Abstract: Technologies are described herein for performing a multi-protocol data transfer. A first protocol that guarantees reliability and in-order delivery of data packets is utilized to transmit control information for a data transfer. Once the control information has been transmitted and received at the destination, a second protocol is utilized to transfer the actual data. The second protocol does not guarantee data reliability or in-order delivery. If any data packets are not delivered to the destination using the second protocol, either the first protocol, the second protocol, or a combination can be utilized to retransmit the missing packets.

Abstract: Technologies are described herein for performing data de-duplication of a version of a data file for backup to a remote storage location. A CDP module executing on a computer creates a collection of files corresponding to the version of the data file by de-duplicating the version against a previous version master file stored locally on the computer. The previous version master file contains one or more unique data blocks of a specific block size from a previous version of the data file. Once the de-duplication against the locally maintained previous version master file is complete, the CDP module stores the collection of files corresponding to the version of the data file to the remote storage location. The remote storage location also contains a master file corresponding to the data file that contains all of the unique data blocks in the previous version master file.

Abstract: Technologies for eliminating duplicate data within a storage system. De-duplication may be performed done at physical chunk level, where the data is not copied or moved to different location. A logical mapping is modified using a thin de-duplication kernel module that resides between a distributed volume manager (DVM) and a logical disk (LD). De-duplication is achieved by changing pointers in the mapping to land at a physical location. De-duplication is performed as post-process feature where duplicates are identified and the duplicates are marked in the mapping table, thereby claiming free space through de-duplication. Block-level de-duplication in accordance with the above can co-exist with existing storage architectures for thin provisioning and snapshot management.

Abstract: Methods and systems for identifying candidates for de-duplication are disclosed herein. An example system may include: a de-duplication lister that coordinates with a kernel module to identify a unit of data for de-duplication and calculates a signature of the unit of data; a de-duplication queue that manages a de-duplication queue; a de-duplication tracker that maintains a de-duplication data structure including a plurality of entries corresponding to units of data and searches the de-duplication data structure to determine whether the signature of the identified unit of data matches the signature of at least one of the entries in the de-duplication data structure; a de-duplication maintainer that reviews the entries of the de-duplication data structure and removes obsolete entries from the de-duplication data structure; and a de-duplication engine that coordinates with the kernel module to perform de-duplication operations on the de-duplication queue.

Abstract: Technologies for eliminating duplicate data within a storage system. De-duplication may be performed done at physical chunk level, where the data is not copied or moved to different location. A logical mapping is modified using a thin de-duplication kernel module that resides between a distributed volume manager (DVM) and a logical disk (LD). De-duplication is achieved by changing pointers in the mapping to land at a physical location. De-duplication is performed as post-process feature where duplicates are indentified and the duplicates are marked in the mapping table, thereby claiming free space through de-duplication. Block-level de-duplication in accordance with the above can co-exist with existing storage architectures for thin provisioning and snapshot management.

Abstract: Technologies are described herein for providing networked RAID in a virtualized storage cluster. The storage capacity of a storage cluster having two or more storage nodes is organized into tiers. A portion of the available storage capacity is allocated to one tier that is organized using chained declustering. Another portion of the available storage capacity is allocated to another tier that is organized using a networked RAID configuration. The storage cluster monitors the frequency at which data in the storage cluster is accessed. Frequently used data that is stored in the network RAID-configured tier is promoted to the chained declustered-configured tier. Infrequently used data that is stored in the chained declustered-configured tier is demoted to the network RAID-configured tier.

Abstract: Technologies are described herein for performing a multi-protocol data transfer. A first protocol that guarantees reliability and in-order delivery of data packets is utilized to transmit control information for a data transfer. Once the control information has been transmitted and received at the destination, a second protocol is utilized to transfer the actual data. The second protocol does not guarantee data reliability or in-order delivery. If any data packets are not delivered to the destination using the second protocol, either the first protocol, the second protocol, or a combination can be utilized to retransmit the missing packets.

Abstract: Technologies are described herein for achieving data consistency during a failover from a primary node to a secondary node in a storage cluster with a shared RAID array in a degraded mode. When the primary storage node receives a write I/O operation, a volume module on the primary node reads the missing data strip data from the failed disk belonging to the stripe targeted by the write I/O operation. The primary storage node communicates the missing data strip to the secondary storage node, which writes the missing strip data to an outstanding strip log. Upon the failure of the primary node, the secondary storage node reads the missing data strip from the outstanding strip log and writes the missing data strip to the shared RAID array, thus restoring data consistency to the stripe of the RAID array containing the missing data strip.

Abstract: A method, system, apparatus, and computer-readable medium are described for the background movement of data between nodes in a storage cluster. According to one method, exception tables are generated that include data identifying the areas on each node that need to be migrated to another node and the areas on each node that are to be migrated from another node. The exception tables may be generated in response to the failure of a node in the storage cluster or in other circumstances. A background resynchronization thread utilizes the contents of the exception tables to migrate data between the nodes of the cluster, thereby synchronizing the data stored on the nodes. Input/output operations directed to an area that is to be migrated from another node are redirected to the other node in accordance with a timeout period associated with the input/output operations.

Abstract: A method, system, apparatus, and computer-readable medium are described for providing distributed hot-spare storage in a redundant storage cluster. According to one method, a portion of the unutilized space on the storage cluster is utilized as a distributed hot-spare storage node. Through this mechanism, a redundant storage cluster with N storage nodes may be contracted to a redundant array with N?1 nodes. Thin provisioning and intelligent data placement may also be utilized to implement the distributed hot-spare storage node. Through repeated application of such methods and systems, the failure of any storage node or the sequential failure of multiple storage nodes within a redundant storage cluster results in the recreation of the cluster as a redundant storage array with one fewer node, but with the same redundancy.

Abstract: Dynamic provisioning of available space in a data storage system without having to configure partitions at system startup is presented. A system table may be maintained with entries corresponding to provisions within the available physical capacity of a data storage system. A volume table may be maintained that includes entries corresponding to territories within a logical data storage volume. When a data write operation is requested, a determination may be made as to whether physical space has been allocated for the territory in the volume that is to be written. If physical space has not yet been allocated, the necessary physical space may be allocated for the territory within the logical volume that is to receive the written data and the requested write operation may be performed. Metadata can be written to disk as to simplify recover from system crashes and unclean shutdowns.

Abstract: Technologies are described herein for providing a power-save mode of operation for a continuous data protection (“CDP”) process. A CDP module executing on a computer detects when the computer is operating under battery power, and enters a power-save mode of operation. In the power-save mode, the CDP module monitors the state of the battery and restricts the CDP process based on a set of configuration settings specified for one of a number of zones, where each zone defines an upper and lower thresholds for a level of charge of the battery. If external power is restored to the computer, the CDP module restores the normal mode of operation for the CDP process.

Abstract: Technologies are described herein for providing networked RAID in a virtualized storage cluster. The storage capacity of a storage cluster having two or more storage nodes is organized into tiers. A portion of the available storage capacity is allocated to one tier that is organized using chained declustering. Another portion of the available storage capacity is allocated to another tier that is organized using a networked RAID configuration. The storage cluster monitors the frequency at which data in the storage cluster is accessed. Frequently used data that is stored in the network RAID-configured tier is promoted to the chained declustered-configured tier. Infrequently used data that is stored in the chained declustered-configured tier is demoted to the network RAID-configured tier. The zone size for zones stored in the chained declustered-configured tier may be different than for zones stored in the network RAID-configured tier.