Abstract: Described are techniques for managing data locations of derived storage objects. The techniques include accessing a primary metadata tree providing a lookup path using a virtual address to locate data for a primary object in physical storage pointed to by a leaf node of the primary metadata tree and providing a sub-tree from a leaf node of the primary metadata tree associated with the primary object, the sub-tree providing lookup paths using virtual addresses for derived objects. Wherein a derived object is a view of the primary object at a specific point in time, with copy leaf nodes of the sub-tree pointing to stored data of the derived objects. The techniques further include maintaining an ancestry graph associated with the sub-tree with the ancestry graph linking derived object identifiers to provide an ancestral chain lookup to identify derived object identifiers of the closest stored physical data of a derived object.

Abstract: Method, apparatus, and computer program product utilizing a hot-spare node in a storage network having a deduplication fingerprints database. A plurality of nodes is provided in the storage network including at least one active node and at least one hot-spare node. A portion of the deduplication fingerprints database is stored on each active node and on each hot-spare node. Data from the deduplication fingerprints database is provided from at least one of said at least one hot-spare node during normal operation. Responsive to a failure of one of said at least one active node, a portion of the deduplication fingerprints database associated with data stored on the failed one of said at least one active node is retained on one of the at least one hot-spare node. The failed one active node is replaced with one of the at least one hot-spare node.

Abstract: An apparatus for improved version updates in data deduplication includes a baseline module that stores a baseline referrer link in a first cohort of baseline referrer links pointing to a first location in storage where a baseline version data block is stored. The apparatus includes a signpost module that stores to metadata a signpost that points to an updated referrer link in a second cohort of updated referrer links that points to a second location in the deduplicating storage device where an updated version data block is stored in response to the updated version data block being generated. The apparatus includes a cohort update module that replaces the baseline referrer link with the updated referrer link in response to determining that an additional updated version data block matches the updated version data block pointed to by the signpost. A method and a system implement functions of the apparatus.

Abstract: A data storage system comprises physical storage, cache memory and a processor connected to the physical storage and the cache memory. The processor is arranged to maintain a set of active regions in the cache memory, each active region having a size equal to an integer multiple of an update size of a flash chip within the physical storage, where the integer could be 1. The processor receives requests for one or more blocks of the cache memory from components within the storage system and allocates one or more blocks from an active region in response to a received request. If the processor determines that all blocks in an active region have been allocated and that all allocated blocks within this region have been written to, then the processor destages the content of this region to the physical storage.

Abstract: A computer-implemented method according to one embodiment includes receiving, by a target system from a source system, a description of a set of data updates that are to be written to the target system. For each given portion of data of the target system that is to be rewritten during performance of the set of data updates, forward lookup is performed on the target system for determining a physical storage address of the given portion of data. The method further includes marking each of the determined physical storage addresses of the portions of data of the target system in a copy of a reverse lookup table of the target system. The marked-up reverse lookup table is used for determining an ordering in which the performance of the set of data updates would result in a least amount of garbage collection being performed while performing the set of data updates.

Abstract: An apparatus for controlling a storage system having a data replication function, comprises: a storage array component being operable to send notification to a replication engine that a write of data to a primary storage location by a host is subject to data replication; the replication engine being operable to receive the notification and in response to instruct the storage array to copy the data to a secondary storage location; wherein the data is copied to the secondary storage location unmediated by the replication engine.

Abstract: A computer-implemented method according to one embodiment includes receiving, by a target system from a source system, a description of a set of data updates that are to be written to the target system. For each given portion of data of the target system that is to be rewritten during performance of the set of data updates, forward lookup is performed on the target system for determining a physical storage address of the given portion of data. The method further includes marking each of the determined physical storage addresses of the portions of data of the target system in a copy of a reverse lookup table of the target system. The marked-up reverse lookup table is used for determining an ordering in which the performance of the set of data updates would result in a least amount of garbage collection being performed while performing the set of data updates.

Abstract: Prioritizing virtual volumes to take offline in a thin provisioning system with garbage collection. The method categorizes virtual volumes based on garbage collection properties of their write behavior and adds metadata indicating a category of a virtual volume. The method schedules virtual volumes to be taken offline by predicting virtual volume space utilization of active virtual volumes for a defined time period in combination with estimated garbage collection in that period to determine a need to take virtual volumes offline. The method selects virtual volumes to take offline by their category to ensure that the virtual volumes producing the most garbage collection unfriendly workloads are taken offline first.

Abstract: An apparatus for improved version updates in data deduplication includes a baseline module that stores a baseline referrer link in a first cohort of baseline referrer links pointing to a first location in storage where a baseline version data block is stored. The apparatus includes a signpost module that stores to metadata a signpost that points to an updated referrer link in a second cohort of updated referrer links that points to a second location in the deduplicating storage device where an updated version data block is stored in response to the updated version data block being generated. The apparatus includes a cohort update module that replaces the baseline referrer link with the updated referrer link in response to determining that an additional updated version data block matches the updated version data block pointed to by the signpost. A method and a system implement functions of the apparatus.

Abstract: A storage drive array with available storage space for user data with storage capacity in the array allocated to distributed spare space. The distributed spare space spreads portions of a spare drive across multiple storage drives in the array. By monitoring the amount of spare space available in the array, a determination may be whether the amount of spare space falls below a threshold number of drives. If there is un-used space in the available storage space which is greater than one drive of space, a drive of space to spare space may be dynamically re-allocated in a distributed pattern across the drives of the array.

Abstract: Method, apparatus, and computer program product utilizing a hot-spare node in a storage network having a deduplication fingerprints database. A plurality of nodes is provided in the storage network including at least one active node and at least one hot-spare node. A portion of the deduplication fingerprints database is stored on each active node and on each hot-spare node. Data from the deduplication fingerprints database is provided from at least one of said at least one hot-spare node during normal operation. Responsive to a failure of one of said at least one active node, a portion of the deduplication fingerprints database associated with data stored on the failed one of said at least one active node is retained on one of the at least one hot-spare node. The failed one active node is replaced with one of the at least one hot-spare node.

Abstract: Data is stored on a primary storage system and a copy of the data is stored on a secondary storage system. A determination is made that a connection between the systems is currently unavailable. Location data is maintained that identifies where changes have been made to the primary storage system while the connection is unavailable. Another determination is made that data has been lost at the secondary storage system. Recovery data required to repair the lost data is identified. Another determination is made that the connection to the secondary storage system is now available. The location data is updated with the locations of the recovery data. The secondary storage system is updated with data from the primary storage system as defined by the location data.

Abstract: A method for maintaining I/O statistics in a tiered storage environment includes maintaining, by a tiered storage layer within a storage system, I/O statistics for data storage elements or areas within the storage system. The tiered storage layer migrates data between storage tiers in accordance with the I/O statistics. A data services layer, that operates independently from the tiered storage layer, alters (e.g., compresses, decompresses) selected data within the storage system and generates a message to the tiered storage layer describing the alteration. The message identifies the selected data, how the selected data before the alteration maps to the selected data after the alteration, and an operator (e.g., percentage) to be applied to I/O statistics associated with the selected data as a result of the alteration The tiered storage layer receives this message and modifies its internal I/O statistics accordingly. A corresponding system and computer program product are also disclosed.

Abstract: A method and a computer system for maintaining synchronization between primary data stored in a primary volume and secondary data stored in a secondary volume. The primary volume and the secondary volume having respective address spaces divided into address blocks, each comprising a range of addresses. The computer system determines a start condition in which the secondary data has been checked as being a mirror of the primary data, transmits writes of the primary data that have occurred since the start condition to the secondary volume, and maintains an incremental record of current differences between the primary and secondary volumes on a per address block basis. The computer system operates a garbage collection algorithm on the secondary volume to clean up the address space.

Abstract: A method for replicating data from a primary site to a secondary site is disclosed. In one embodiment, such a method includes tracking, at a primary site, updates to storage elements of a primary volume over a specified period of time. The method further calculates a size associated with the updates and communicates the size from the primary site to a secondary site. At the secondary site, using the size received from the primary site, the method determines an amount of storage space required to accommodate the updates in a secondary volume. In certain embodiments, this may include taking into account compression or deduplication algorithms at the secondary site. The method allocates the storage space to the secondary volume and replicates, from the primary site to the secondary site, the updates to the secondary volume. A corresponding system and computer program product are also disclosed.

Abstract: A computer-implemented method of determining modified portions of a RAID storage array for use in resynchronizing said RAID storage array after a failure, the computer-implemented method comprising: resolving areas in the RAID storage array that represent space allocated to volumes; resolving which of said allocated volumes comprise gathered writes; and for said allocated volumes that comprise gathered writes, resolving a set of writes that potentially have incomplete parity updates at the time of the failure.

Abstract: A mechanism is provided for backing up metadata in a data storage system. Given metadata stored on the data storage system is implemented by way of a tree structure operable by fixed mapping. A stream of snapshots is taken of leaves in the metadata tree structure of given metadata stored on persistent data storage. The metadata tree structure of the given metadata stored on the persistent data storage is reconstructed on a basis of the stream of snapshots taken for backing up the given metadata stored on the persistent data storage. A journal over the reconstructed metadata tree structure is then replayed. In response to a given status discrepancy being flagged in respect of any given metadata, the status of a given leaf corresponding thereto in the reconstructed data tree structure is modified for consistency with the corresponding journal entry in the journal for that given metadata.

Abstract: Cache management. It is determined that a cache operation relating to the transfer of data between a cache memory and a data storage system is required. A state of a utilization model is received, the utilization model including requirements for utilization of resources of the data storage system over a time period, and the state indicating a cost of resource utilization associated with cache operations in the current time period. It is determined whether to perform the cache operation, based on the utilization requirements and the state of a utilization model. If the cache operation is not to be performed, and if the cache operation is a write operation, it is determined whether the cache memory is full. If so, the cache operation is managed according to an emergency cache management process; if not, the data associated with the cache operation is maintained in the cache memory.

Abstract: A method and a computer system for maintaining synchronization between primary data stored in a primary volume and secondary data stored in a secondary volume. The primary volume and the secondary volume having respective address spaces divided into address blocks, each comprising a range of addresses. The computer system determines a start condition in which the secondary data has been checked as being a mirror of the primary data, transmits writes of the primary data that have occurred since the start condition to the secondary volume, and maintains an incremental record of current differences between the primary and secondary volumes on a per address block basis. The computer system operates a garbage collection algorithm on the secondary volume to clean up the address space.

Abstract: Method and system are provided for storage capacity allocation. The method includes: providing a storage pool having multiple storage drive arrays; designating an initial available storage capacity in the storage pool; and allocating the remaining storage capacity in the storage pool to distributed spare space, wherein distributed spare space spreads portions of a spare drive across multiple storage drives in an array. The method also includes: monitoring an amount of available storage capacity as data is stored to the storage pool and determining when a threshold of a minimum available storage capacity is reached; and re-allocating one or more distributed spare drives from an array to available storage capacity when the threshold of the minimum available storage capacity is reached.