Abstract: Methods, apparatus and computer program products implement embodiments of the present invention that enable digital information devices having respective storage devices and memories to distributedly store, for a logical volume, data and first and second parity values across corresponding regions of the storage devices. Freezers having a one-to-one correspondence with the storage devices are distributedly stored in the memories. Upon detecting, in a cache, updated data for one or more first regions on the storage devices, existing data from the one or more first regions, and additional data for parity calculations are retrieved from one or more corresponding second regions on the storage devices, and first and the second parity values are calculated using the updated data, the existing data and the additional data, and the updated data and the calculated first and second parity values are stored to the freezers.

Abstract: Methods, apparatus and computer program products implement embodiments of the present invention that include defining, in a storage system including receiving, by a storage system, a storage request, and identifying, based on the storage request, one or more storage management units. For each of the storage management units, a master partition table having multiple master entries and one or more backup partition tables is identified, each of the backup partition tables having backup entries in a one-to-one-correspondence with the master entries. Upon detecting a failure in master partition table, one of the backup partition tables is configured as a replacement master partition table, essential metadata is used to calculate non-essential metadata in the replacement master partition table, and the calculated non-essential metadata is stored to the replacement master partition table.

Abstract: Embodiments of the invention relate to data placement for loss protection in a storage system. One embodiment includes constructing multiple logical compartments. Each logical compartment includes a placement policy including a set of storage placement rules for determining permitted placement of storage symbols on specific physical storage containers, and a balancing policy for determining a particular placement of the storage symbols among permitted placements for balancing the storage symbols for each volume among the physical storage containers that is useable within the placement policy.

Abstract: Embodiments relate to host-side cache migration. An aspect is a method that includes determining pre-fetch hints associated with a source cache that is local to a source host machine. The source cache includes pages of cache data for a virtual machine. The pre-fetch hints are sent to a pre-fetch planner to create a pre-fetch plan. The pre-fetch hints are sent based on migration of the virtual machine to a target host machine including a target cache that is local. At the source host machine, a cache migration request is received based on the pre-fetch plan. A first subset of the pages is sent from the source cache through a host-to-host communication channel to the target cache based on the cache migration request. A second subset of the pages is sent from the source cache through a host-storage communication channel to a shared storage to be relayed to the target cache.

Abstract: Methods, apparatus and computer program products implement embodiments of the present invention that include defining, in a storage system including multiple storage devices, multiple redundant array of independent disks (RAID) configurations for data stored on the storage devices. A first one of the multiple RAID configurations is assigned to a first entry in a partition table having multiple entries. Assigning the first one of the multiple RAID configurations to the first entry in the partition table comprises modifying an existing RAID configuration for the first entry. Data referenced by the existing RAID configuration is migrated to the modified RAID configuration.

Abstract: Methods, apparatus and computer program products implement embodiments of the present invention that include identifying non-destaged first data in a write cache. Upon detecting second data in a master read cache, the second data is pinned to the master and one or more backup read caches. Using the first data stored in the write cache and the second data stored in the master read cache, one or more parity values are calculated, and the first data and the one or more parity values are destaged.

Abstract: Methods, apparatus and computer program products implement embodiments of the present invention that include defining, in a storage system including receiving, by a storage system, a storage request, and identifying, based on the storage request, one or more storage management units. For each of the storage management units, a master partition table having multiple master entries and one or more backup partition tables is identified, each of the backup partition tables having backup entries in a one-to-one-correspondence with the master entries. The storage request is performed, and upon the storage request being performed, any changes to essential metadata in the one or more master partition tables are identified, and the identified changes are stored to the essential metadata in the one or more backup partition tables.

Abstract: Migrating a sub-volume in data storage with at least two de-duplication domains, each of the domains having at least one sub-volume. A first sub-volume is assigned to a de-duplication domain and a first content summary is computed for the first sub-volume. Similarly, a second sub-volume is assigned to a second de-duplication domains and a second content summary is computed for the second sub-volume. A first content affinity is calculated between the first sub-volume and a third sub-volume, and a second content affinity is calculated between the second sub-volume and the third sub-volume. A domain placement is selected for the third sub-volume based on comparison of the first content affinity and the second content affinity.

Abstract: Methods, apparatus and computer program products implement embodiments of the present invention that include defining, in a storage system including multiple storage devices, multiple redundant array of independent disks (RAID) configurations for data stored on the storage devices. A first one of the multiple RAID configurations is assigned to a first entry in a partition table having multiple entries, and a second one of the multiple RAID configurations is assigned to a second entry in the partition table, the second one of the multiple RAID configurations independent from the first one of the multiple RAID configurations.

Abstract: Methods, apparatus and computer program products implement embodiments of the present invention that include identifying non-destaged first data in a write cache. Upon detecting second data in a master read cache, the second data is copied the second data to one or more backup read caches, and the second data is pinned to the master and the backup read caches. Using the first data stored in the write cache and the second data stored in the master read cache, one or more parity values are calculated, and the first data and the one or more parity values are destaged.

Abstract: Assignment of files to a de-duplication domain. Address space of data files is divided into multiple containers. For each of the containers, a file metadata scan is performed to obtain file system metadata, which is aggregated and summarized in a content feature summary. A content feature summary prediction measurement is measured between containers from the generated content feature summary, and files from each container are assigned to a de-duplication domain based upon the content similarity predication measurement.

Abstract: Assignment of files to a de-duplication domain. Address space of data files is divided into multiple containers. For each of the containers, a file metadata scan is performed to obtain file system metadata, which is aggregated and summarized in a content feature summary. A content feature summary prediction measurement is measured between containers from the generated content feature summary, and files from each container are assigned to a de-duplication domain based upon the content similarity predication measurement.

Abstract: A protocol is employed to estimate duplication of data in a storage system. This estimate is employed as a factor of enabling de-duplication, and if de-duplication is enabled, the data sets which will be subject to the de-duplication. The protocol includes a measurement procedure and an execution procedure. The measurement procedure characterizes data duplication in part of the data on the storage system, and the execution procedure use the characterization to adjust selection of which data sets are subject to de-duplication.

Abstract: Embodiments of the invention relate to compressed storage systems, and reducing metadata representing compressed data. Compressed data is stored in units referred to as partitions, with each partition having a header that contains a virtual address of data stored in the partition. A linear function is providing to represent a mapping between a virtual address segment and a compressed data extent, with a slope of the function representing an associated compression ratio. A read operation is supported by consulting the mapping and using the mapping to locate the corresponding compressed extent. Similarly, a write operation is supported by writing a new segment, compressing content in the segment, and computing a new mapping of the compressed segment metadata in memory. The new mapping is represented in the linear function.

Abstract: Embodiments of the invention relate to storage systems, and for recording event records into a log. Different statistics may be tallied from the log, with the different statistics generating different statistical measurements. The aspect of processing of I/O and preparation of statistical measurements are separated, thereby enabling data structures and algorithms which would not be considered feasible for use as part of I/O processing, because they may compromise the performance or other essential characteristic of I/O processing, to be employed.

Abstract: Detecting data duplication includes maintaining a fingerprint directory including one or more entries. Each entry includes a data fingerprint and a data location for a data chunk. A shadow list including a record of fingerprint values not contained in the fingerprint directory is maintained. Each entry is associated with a seen-count attribute, which is an indication of how often a data fingerprint has been seen in arriving data chunks to be written in a storage system, and distinguishes multiply-seen entries for data fingerprints present in at least two data chunks from once-seen entries for data fingerprints present in no more than a single data chunk. Each entry retrieved from the shadow list relates to twice-seen fingerprints.

Abstract: Migrating a sub-volume in data storage with at least two de-duplication domains, each of the domains having at least one sub-volume. A first sub-volume is assigned to a de-duplication domain and a first content summary is computed for the first sub-volume. Similarly, a second sub-volume is assigned to a second de-duplication domains and a second content summary is computed for the second sub-volume. A first content affinity is calculated between the first sub-volume and a third sub-volume, and a second content affinity is calculated between the second sub-volume and the third sub-volume. A domain placement is selected for the third sub-volume based on comparison of the first content affinity and the second content affinity.

Abstract: Embodiments of the invention relates to avoiding out-of-space conditions in storage controllers operating with efficiency capabilities between virtual space in a data container and real space in a storage container. Both the real space and the virtual space are monitored and their respective usage is compared to provide information about occupancy of the real space to the virtual space. Usage of the containers is balanced by employing a virtual file associated with a reserved portion of free capacity in the virtual space.

Abstract: Detecting data duplication comprises maintaining a fingerprint directory including one or more entries, each entry including a data fingerprint and a data location for a data chunk. Each entry is associated with a seen-count attribute which is an indication of how often the fingerprint has been seen in arriving data chunks. Higher-frequency entries in the directory are retained, while also taking into account recency of data accesses. A data duplication detector detects that the data fingerprint for a new chunk is the same as the data fingerprint contained in an entry in the fingerprint directory.

Abstract: A data storage system comprising a Quality of Service server to optimize revenue realized under multiple service level agreements with a data storage client.