Abstract: Described herein are systems and methods for performing TRIM commands in a data storage system. An example data storage system can include physical storage including a solid state drive (SSD), and a storage system computer operably coupled to the physical storage. The storage system computer can include a processing unit and a memory operably coupled to the processing unit. The data storage system can further include a distributed volume management (DVM) module stored in the memory that, when executed by the processing unit, causes the processing unit to: allocate a logical volume from the physical storage; subdivide the logical volume into a plurality of logical blocks; maintain a mapping table for tracking the logical blocks of the logical volume; and in response to a predetermined event, execute a TRIM command for one or more unused data blocks of the SSD.

Abstract: Systems, methods, and computer-readable media for handling I/O operations in a storage system are described herein. An example method includes assigning each of a plurality of storage devices to one of a plurality of tiers; imposing a hierarchy on the tiers; creating a logical volume by reserving a portion of a storage capacity for the logical volume without allocating the portion of the storage capacity to the logical volume; and assigning the logical volume to one of a plurality of volume priority categories. The method includes receiving a write I/O operation directed to a logical unit of the logical volume; and allocating physical storage space for the logical unit of the logical volume in response to the write I/O operation. The physical storage space is located in one or more storage devices. The method includes writing data associated with the write I/O operation to the one or more storage devices.

Abstract: Systems and methods for handling input/output operations during a space crunch are described herein. An example method includes striping a volume across a plurality of storage nodes, and maintaining a cluster volume table (CVT) storing information regarding distribution of the logical blocks. Additionally, the CVT includes a plurality of entries, where each of the entries includes information identifying a respective owner storage node of a respective logical block. The method also includes receiving a write I/O operation directed to an unallocated logical block owned by a landing storage node, where the landing storage node lacks free storage capacity, and locking the unallocated logical block. The method further includes updating the CVT to identify a storage node having free storage capacity as owner storage node of the unallocated logical block, and unlocking the unallocated logical block, wherein the write I/O operation proceeds at the storage node having free storage capacity.

Abstract: A method for determining when to load read I/O operations into an SSD cache medium for a physical storage medium of a data storage system can include maintaining an SSD filter bitmap with a plurality of bits, where each of the bits corresponds to a respective data block of the physical storage medium. The method can also include initially setting each of the bits to a first predetermined value, receiving a first read I/O operation directed to a particular data block of the physical storage medium and, in response to receiving the first read I/O operation, setting a bit corresponding to the particular data block to a second predetermined value. The method can further include receiving a second read I/O operation directed to the particular data block and, in response to receiving the second I/O operation, loading data for the particular data block into the SSD cache medium.

Abstract: Described herein are systems, methods, and computer-readable media for providing enhanced encryption in a data storage system. An example method can include receiving a data set, selecting a first portion of the data set as a unique encryption key, encrypting a second portion of the data set using the unique encryption key, and writing the encrypted second portion of the data set to a storage device.

Abstract: Technologies are described herein for accelerating the boot process of client computers by consolidating client-specific boot data in a data storage system. Boot statistics are collected for a number of client computers booting from virtual storage volumes provided by the data storage system. The boot statistics are analyzed to identify client-specific boot data stored on each of the virtual storage volumes, and the client-specific boot data is consolidated and copied into contiguous regions of a single, consolidated boot volume in the data storage system. Requests for read operations from the client computers for the client-specific boot data are then redirected to the consolidated boot volume, increasing boot performance for the client computers.

Abstract: Systems, methods, and computer-readable media for intelligent distribution of data in a storage cluster are described herein. An example method includes maintaining a cluster volume table (CVT) that stores information regarding data distribution of a volume across a plurality of storage nodes, and defining data distribution groups within the CVT. The CVT includes a plurality of entries, where each entry identifies an owner storage node for a respective logical block of the volume. Each data distribution group includes consecutive CVT entries. The method also includes receiving an input/output (I/O) operation directed to a region of the volume, identifying an owner storage node associated with the region of the volume using the CVT, and serving the I/O operation with at least one storage node of a data distribution group associated with the owner storage node. The I/O operation is served at a chunk-level granularity that is finer than a logical-block-level granularity of the volume.

Abstract: Systems and methods for consolidating space in a storage cluster are described herein. An example method can include striping a volume across a plurality of storage nodes, and maintaining a cluster volume table (CVT) storing information regarding distribution of a plurality of logical blocks across the storage nodes. The CVT can include a plurality of entries, where each of the entries can include information identifying a respective owner storage node of a respective logical block. The method can also include adding a new storage node to the storage cluster, and transferring data for one or more logical blocks from each of the storage nodes to the new storage node based on respective usage ratios of the storage nodes. The method can further include updating the CVT to identify the new storage node as owner storage node of the one or more logical blocks transferred to the new storage node.

Abstract: Systems, methods, and computer-readable media for intelligent distribution of data in a storage cluster are described herein. An example method can include striping a volume across a plurality of storage nodes. For example, a plurality of logical blocks of the volume can be distributed to the storage nodes in relation to respective sizes of the storage nodes. Additionally, the method can include maintaining a cluster volume table (CVT) storing information regarding distribution of the logical blocks across the storage nodes. The CVT can include a plurality of entries, where each of the entries in the CVT can include information identifying a respective owner storage node of a respective logical block. Optionally, a factor can be used to determine a number of logical blocks distributed to a storage node for each stripe when striping the volume across the storage nodes.

Abstract: A computer-implemented method for predicting a risk of disk failure is described herein. The method can include collecting a performance parameter for respective disks in a group of disks, and calculating an expected value of the performance parameter for the group of disks. The method can also include comparing a respective value of the performance parameter for a respective disk in the group of disks to the expected value, and predicting a risk of disk failure based on the comparison. For example, when a difference between the respective value of the performance parameter for the respective disk and the expected value exceeds a threshold, the respective disk may be at risk of failing.

Abstract: An example computer-implemented method for restoring data consistency in a RAID array can include detecting failure of a disk of the RAID array, maintaining tracking information for write input/output (“I/O”) operations that effect a data block or a parity block stored on the failed disk, re-commissioning the failed disk and re-synchronizing the RAID array using the tracking information. The tracking information can be used to restore consistency to at least one of the data stripes. For example, the RAID array can store data in one or more data stripes, where each data stripe has a plurality of data and parity blocks. Additionally, the data stripes to which consistency is restored can include a data block or a parity block that is stored on the re-commissioned disk and that is effected by the write I/O operations and stored on the re-commissioned disk.

Abstract: An example computer-implemented method for performing failover operations in a data storage system is described herein. The data storage system can include a first storage controller and a second storage controller for processing input/output (“I/O”) operations for the data storage system. The method can include, in response to a failure of the first storage controller, performing failover operations with the second storage controller, and processing the I/O operations with the second storage controller. The failover operations can include preparing a disk subsystem layer for I/O operations, preparing a device manager layer for the I/O operations, and preparing a network layer for the I/O operations. The disk subsystem, device manager, and network layers can be prepared for the I/O operations without dependencies. In particular, preparation of the network layer is not dependent on preparation of the disk subsystem layer or the device manager layer.

Abstract: An example method for performing cache replacement in a caching medium for a data storage system can include providing an SSD cache, providing an LRU data structure including buckets for managing the SSD cache, and providing cache headers for managing the cache lines. The method can include assigning two or more cache headers to a same bucket of the LRU data structure, and arranging the cache headers in a linked list based on access time. A cache header for an LRU cache line is a tail node of the linked list. The method can further include providing an LFU data structure including frequency buckets, assigning the tail node of the linked list of the same bucket of the LRU data structure to a frequency bucket based on access frequency, and selecting an LFU cache line for cache replacement using the LFU data structure.

Abstract: Methods and systems for identifying candidates for de-duplication are disclosed herein. An example system may include: a de-duplication lister that coordinates with a kernel module to identify a unit of data for de-duplication and calculates a signature of the unit of data; a de-duplication queue that manages a de-duplication queue; a de-duplication tracker that maintains a de-duplication data structure including a plurality of entries corresponding to units of data and searches the de-duplication data structure to determine whether the signature of the identified unit of data matches the signature of at least one of the entries in the de-duplication data structure; a de-duplication maintainer that reviews the entries of the de-duplication data structure and removes obsolete entries from the de-duplication data structure; and a de-duplication engine that coordinates with the kernel module to perform de-duplication operations on the de-duplication queue.

Abstract: A method for offloading RMW operations on a volume of a storage system to a caching medium includes providing an SSD cache, a data structure for managing the SSD cache and cache headers for managing cache lines. The method also includes maintaining cache headers for cache lines corresponding to a same chunk of the volume as a group, creating a snapshot of the volume and, after creating the snapshot, receiving a write I/O operation directed to a portion of the same chunk. The method further includes allocating a new RMW header for the write I/O operation, maintaining the new RMW header in a new group, performing the write I/O operation on the SSD cache, allocating an old RMW header for the group, maintaining the old RMW header with the group and providing a link between the new and old RMW headers.

Abstract: A method for determining when to load read I/O operations into an SSD cache medium for a physical storage medium of a data storage system can include maintaining an SSD filter bitmap with a plurality of bits, where each of the bits corresponds to a respective data block of the physical storage medium. The method can also include initially setting each of the bits to a first predetermined value, receiving a first read I/O operation directed to a particular data block of the physical storage medium and, in response to receiving the first read I/O operation, setting a bit corresponding to the particular data block to a second predetermined value. The method can further include receiving a second read I/O operation directed to the particular data block and, in response to receiving the second I/O operation, loading data for the particular data block into the SSD cache medium.

Abstract: Disclosed herein is an enhanced volume manager (VM) for a storage system that accelerates input/output (I/O) performance for random write operations to a striped disk array using parity. More specifically, various implementations are directed to accelerating “random writes” (writes comprising less than a complete stripe of data) by consolidating several random writes together to create a “sequential write” (a full-stripe write) to eliminate one or more read operations and/or increase the volume of new/updated data stored for each write operation. Several such implementations comprise functionality in the VM (volume manager) for identifying random write I/O requests, queuing them locally in a journal, and then periodically flushing the journal to the disk array as a sequential write request.

Abstract: A method for managing a caching medium for a storage system includes providing an SSD cache including a plurality of cache lines, providing a data structure including a plurality of buckets for managing the SSD cache, and providing a plurality of cache headers for managing the cache lines. Each cache line has a first predetermined size, and each bucket corresponds to a contiguous region of a physical storage capacity having a second predetermined size. Each cache header associates a cache line and a corresponding data block stored in the data storage system. The method also includes assigning two or more cache headers for cache lines associated with corresponding data blocks stored in a same contiguous region of the physical storage capacity to a same bucket, and maintaining the two or more cache headers as a first group of cache headers within the same bucket.

Abstract: An example method for using a plurality of SSDs as a caching medium in a data storage system can include splitting the SSDs into a plurality of cache regions, maintaining a first cache region of the SSDs as a read cache and maintaining a second cache region of the SSDs as a write cache. Additionally, read data can be stored by striping the read data across the SSDs in the first cache region, and dirty write data can be stored by mirroring the dirty write data across the SSDs in the second cache region. The computer-implemented method can also include dynamically adjusting a size of at least one of the first and second cache regions of the SSDs based on an input/output (“I/O”) load of the data storage system.

Abstract: An example computer-implemented method for performing efficient patrol read operations in a storage system including a plurality of disks organized in a RAID array can include determining an I/O load on the storage system, identifying at least one portion of an available storage capacity containing valid data and reading the portion of the available storage capacity containing the valid data. Optionally, the portion of the available storage capacity containing the valid data is the only portion read during the patrol read operations. The method can also include determining whether a medium error exists in the portion of the available storage capacity containing the valid data. If a medium error exists, the method can include fixing the medium error. Additionally, the above operations can optionally be performed only when the I/O load on the storage system is less than a predetermined threshold.