Abstract: Techniques for handling data with different lifetime characteristics in stream-aware data storage systems. The data storage systems can include a file system that has a log-based architecture design, and can employ one or more solid state drives (SSDs) that provide log-based data storage, which can include a data log divided into a series of storage segments. The techniques can be employed in the data storage systems to control the placement of data in the respective segments of the data log based at least on the lifetime of the data, significantly reducing the processing overhead associated with performing garbage collection functions within the SSDs.

Abstract: Allocation of parity data storage extents for a virtual RAID is disclosed. A virtual RAID can be enabled by a virtual storage device pool (VSDP) mapped to a physical storage device pool (PSDP) that can comprise physical extents of physical disks. In an aspect, the physical disks can be co-located, remotely located, or combinations thereof. Mapping of the VSDP can enable allocation of virtual extents in a virtual RAID that can, for example, emulate a RAID4 while still providing parity data storage diversity that can, for example, emulate RAID5, RAID6, etc. Moreover, the disclosed subject matter can support proactive wear leveling, for example, based on historical storage of parity data via an extent, e.g., extents previously used to store parity data. Furthermore, the disclosed subject matter can support active wear leveling.

Abstract: A deduplication index is generated having multiple entries, each entry storing a digest of a data block that was previously stored in non-volatile data storage together with a pointer to the location in non-volatile storage at which the data block was previously stored. The entries of the disclosed deduplication index are divided into multiple deduplication index segments. A resident subset of the deduplication index segments is stored in memory of the data storage system. A non-resident subset of the deduplication index segments is stored in non-volatile data storage of the data storage system. Data deduplication is performed for each subsequently received data block for which a digest is generated that matches any one of the digests in the entries of the deduplication index segments that are contained in the resident subset of the deduplication index segments.

Abstract: A method for storing data in a system that includes a plurality of storage devices, the method that includes obtaining object usage data from the plurality of storage devices, determining, using the object usage data, object clusters, where at least one object cluster of the object clusters includes at least two objects that are associated based on access patterns, migrate a first object, of the two objects, from a first storage device of the plurality of storage devices to a second storage device of the plurality of storage devices.

Abstract: A method, computer program product, and computer system for identifying, by a computing device, information associated with a relationship between a physical layer block and a virtual logic block for RAID storage. The information associated with the relationship between the physical layer block and the virtual logic block may be written within the RAID storage. The physical layer block within the RAID storage may be rebuilt only when the physical layer block includes the information associated with the relationship between the physical layer block and the virtual logic block.

Abstract: A method for provided use in a storage device having a controller, the method comprising: identifying, by the controller, a plurality of logical regions in the storage device; obtaining, by the controller, a respective usage metric for each of the logical regions; updating, by the controller, a translation data structure of the storage device, the translation data structure being updated to map any of the logical regions of the storage device to a respective physical portion of the storage device, the respective physical portion being selected based on the respective usage metric of the logical region, wherein the translation data structure is part of a flash translation layer of the storage device, and the translation data structure is configured to store mapping information between a logical address space of the storage device and a physical address space of the storage device.

Abstract: Techniques for data placement may include receiving data portions stored at logical addresses, and storing the data portions on slices of physical storage located in storage tiers. The storage tiers may include different size slices on the different tiers. In one embodiment, slices in the same tier are all the same size. In another embodiment, slices in the same tier may be of different sizes. Slices of data may be demoted and promoted among the storage tiers as the workloads of the slices changes over time. Demotion may include combining slices into a larger slice. Promotion may include partitioning a slice into smaller slices. Additionally, multiple slices of a tier may be combined into a larger slice in the tier. A slice in the tier may be partitioned into multiple smaller slices also located in the tier.

Abstract: Allocation of parity data storage extents for a virtual RAID is disclosed. A virtual RAID can be enabled by a virtual storage device pool (VSDP) mapped to a physical storage device pool (PSDP) that can comprise physical extents of physical disks. In an aspect, the physical disks can be co-located, remotely located, or combinations thereof. Mapping of the VSDP can enable allocation of virtual extents in a virtual RAID that can, for example, emulate a RAID4 while still providing parity data storage diversity that can, for example, emulate RAID5, RAID6, etc. Moreover, the disclosed subject matter can support proactive wear leveling, for example, based on historical storage of parity data via an extent, e.g., extents previously used to store parity data. Furthermore, the disclosed subject matter can support active wear leveling.

Abstract: Drive health information is collected for each one of the data storage drives in one or more RAID groups of data storage drives, and is used to calculate a faultiness level of each data storage drive in the RAID group(s). A suitable RAID level for configuration of at least one RAID group may be generated based on the faultiness levels of the data storage drives contained in the RAID group. A faultiness-balanced distribution of the data storage drives across multiple RAID groups may be generated based on the faultiness levels of individual data storage drives. The data storage drives may be automatically redistributed across the multiple RAID groups according to the faultiness-balanced distribution of the data storage drives.

Abstract: Techniques for processing I/O operations may include: receiving, at a data storage system, a write operation that writes first data to a target logical address of a log, wherein the data storage system includes a first storage tier of rotating non-volatile storage devices and a second tier of non-volatile solid state storage devices; storing the first data of the target logical address in a first level cache; destaging the first data from the first level cache to a first physical storage location in the first storage tier; and determining, in accordance with first read activity information for the target logical address, whether to store the first data for the target logical address in a second level cache including at least a portion of the non-volatile solid state storage devices of the second tier. The second level cache is a content addressable caching layer that caches data based on read activity.

Abstract: A mapped RAID system with a protection group of physical data storage drives calculates an affinity value for each pair of physical data storage drives in the protection group that is equal to a total number of RAID extents to which at least one drive extent is allocated from each of the physical data storage drives in the pair. A change from a previous level of RAID protection to a new level of RAID protection provided by one of the RAID extents is detected. The new level of RAID protection provided by the RAID extent uses fewer drive extents than the previous level of RAID protection provided by the RAID extent. The disclosed technology selects one of the drive extents previously allocated to the RAID extent for deallocation such that the range of the affinity values is minimized after deallocation of the drive extent.

Abstract: A method is used in managing deduplication of data in storage systems. A candidate data object is identified for deduplicating a data object by evaluating digests stored in a current digest segment to determine whether another digest matching a digest associated with the data block is stored in the current digest segment. The current digest segment includes a set of digests associated with a set of data blocks previously received for deduplication. Based on the evaluation, a deduplicating technique is applied to the data object. The current digest segment is stored in an index table. A previous digest segment associated with a digest stored in the index table matches the digest associated with the data block is replaced by the current digest segment.

Abstract: A method for storing data in a system that includes a plurality of storage devices, the method that includes obtaining object usage data from the plurality of storage devices, determining, using the object usage data, object clusters, where at least one object cluster of the object clusters includes at least two objects that are associated based on access patterns, migrate a first object, of the two objects, from a first storage device of the plurality of storage devices to a second storage device of the plurality of storage devices.

Abstract: A method, computer program product, and computing system for receiving telemetry data associated with a plurality of storage devices. A storage device reliability level may be defined for each storage device based upon, at least in part, the telemetry data associated with the plurality of storage devices. Each storage device may be partitioned into a plurality of storage device segments. A storage device segment reliability level may be defined for the plurality of storage device segments based upon, at least in part, the storage device reliability level of the plurality of storage devices. A subset of the plurality of storage device segments may be combined, thus defining a storage device segment extent. A RAID level for the storage device segment extent may be selected based upon, at least in part, the storage device segment reliability level of each storage device segment of the storage device segment extent.

Abstract: Mapped RAID protection is provided for data written into multiple logical RAID spaces, and the data written to each one of the RAID spaces is stored in data storage drives contained in a corresponding one of multiple protection groups of data storage drives. Drive health information is collected for each data storage drive contained in the protection groups, and a faultiness level of each individual data storage drive in the protection groups is calculated based on the drive health information. A faultiness-balanced distribution of the data storage drives across the protection groups is then generated based on the calculated faultiness levels of the individual data storage drives.

Abstract: A method for use in a computing system, the method comprising: identifying plurality of storage devices; generating a plurality of virtual drive sets, each virtual drive set corresponding to a different one of the plurality of storage devices, each virtual drive set including a plurality of virtual drives, such that each of the virtual drives in the virtual drive set is mapped to a different portion of the virtual drive set's corresponding storage device; instantiating a plurality of storage drive arrays, each of the storage drive arrays being formed of virtual drives from different virtual drive sets; instantiating a plurality of logical units, each of the logical units being instantiated on a different one of the plurality of storage drive arrays; and assigning at least some of the logical units to different software components that are executed on the computing system.

Abstract: A method, computer program product, and computing system for compartmentalizing a plurality of RAID extents, within a RAID system, into a plurality of rotation subgroups. A first logical data portion is written to a first rotation subgroup chosen from the plurality of rotation subgroups. A wear imbalance condition is sensed in a first rotation subgroup, chosen from the plurality of rotation subgroups, that is associated with a plurality of logical data portions.

Abstract: A method, computer program product, and computing system for compartmentalizing a plurality of RAID extents, within a RAID system, into a plurality of rotation subgroups. A write load imbalance condition is sensed in a first rotation subgroup, chosen from the plurality of rotation subgroups, that is associated with a plurality of logical data portions. At least one of the plurality of logical data portions is moved from the first rotation subgroup to a second rotation subgroup, chosen from the plurality of rotation subgroups.

Abstract: A method, computer program product, and computing system for compartmentalizing a plurality of RAID extents, within a RAID system, into a plurality of rotation subgroups. An IO load imbalance condition is sensed in a first rotation subgroup, chosen from the plurality of rotation subgroups, that is associated with a plurality of logical data portions. At least one of the plurality of logical data portions is moved from the first rotation subgroup to a second rotation subgroup, chosen from the plurality of rotation subgroups.

Abstract: A method, computer program product, and computing system for compartmentalizing a plurality of RAID extents, within a RAID system, into a plurality of rotation subgroups. An IO overload condition is sensed in at least one drive extent associated with a first rotation subgroup, chosen from the plurality of rotation subgroups. Instructions are provided concerning moving at least a portion of a load experienced by the first rotation subgroup to a second rotation subgroup, chosen from the plurality of rotation subgroups.