Abstract: Systems and methods for replicating data are disclosed. Data stored in a compressed form on a source storage array or device can be read and transmitted to a destination storage array or device. The replication of data is achieved without having to decompress the data.

Abstract: A system, computer program product, and computer-executable method of providing a layout-independent cryptographic stamp of a distributed data set from a data storage system, the system, computer program product, and computer-executable method comprising receiving a request for a cryptographic stamp of the distributed data set, creating a hash of each slice of the distributed data set, and using each hash from each slice of the distribute data set to create the cryptographic stamp of the distributed data set.

Abstract: Determining an expected compression rate for a prospective process in a federated system includes obtaining compression rate data for existing processes in the federated system, compiling the compression rate data into a plurality of entries in a process name table according to process identifier, client, and industry, determining a specific entry in the process name table for an existing process that most closely matches the prospective process, and determining an expected compression rate of the prospective process based on the compression rate data for the specific entry. Compression rate data may be provided by a driver at host systems that sends compression rate information to a central repository. The central repository may be provided by a host system at a data center of the federated system. The compression rate data may use a sliding average that weighs the data more heavily to favor more recent data.

Abstract: Systems and methods for scheduling multiple flows in a computing system or for allocating resources to the flows in the computing system. Each flow may be associated with a target priority and a target performance. A current priority can be determined for each flow and resources are allocated based on the highest current priority. Over time, the current priorities change and each flow will receive resources as the flows are scheduled for execution. No flow is starved of resources.

Abstract: Example embodiments of the present invention relate to methods, systems, and a computer program product for performing asynchronous replication of a snapshot. The method includes assigning a unique identifier to a snapshot at a source node in a data storage system. Asynchronous replication of the snapshot, including the unique identifier, then may be performed from the source node to a target node. Integrity of the replicated snapshot having the unique identifier then may be verified at the target node.

Abstract: Described embodiments provide systems and methods for operating a storage system. A portion of data currently on a source Logical Unit (LUN) of a disk array of a plurality of disk arrays is written to a destination LUN of a disk array of a plurality of disk arrays. The destination LUN of a disk array of the plurality of disk arrays to write the data to is established based on an efficiency indicator associated with the destination LUN of a disk array of the plurality of disk arrays. The writing at least a portion of data currently on a source LUN of a disk array of a plurality of disk arrays may be performed as part of a replication process or as part of a relocation process.

Abstract: Described embodiments provide systems and methods for low disk space notification. Data modules maintain space usage counters for normal stripes and for checkpoint stripes. The data modules also monitor in use space counters; free space counters; and checkpoint space counters. A system wide total space is calculated, as well as in use space, free space and checkpoint space by aggregating worst case space usage from all data modules. A checkpoint free space ratio and an overall system free space ratio are determined and a checkpoint notification is triggered when the checkpoint free space ratio is lower than a first threshold or when the overall system free space ratio is lower than a second threshold. Upon a checkpoint notification being triggered, a checkpoint notification handling routine is executed to free up disk space.

Abstract: Described embodiments provide systems and methods for online system recovery orchestration. Embodiments may include receiving a request to recover a system management process from a checkpoint. The system management initialization parameter file is updated by setting a flag to start from the checkpoint. A system management process is started with the updated initialization parameter and a copy of the system management repository is loaded. Volumes are recovered before data path module activation. Data modules are then activated with checkpoint offset information and data module journals are reformatted. Control modules are then activated with recovered volume configuration and control module journals are reformatted. Defragmentation mode is set to rollback mode and data on checkpoint stripes that were written after checkpoint creation are deleted. After waiting for the defragmentation rollback to complete the host IO direction is set to checkpoint stripes.

Abstract: A system, computer program product, and computer-executable method of providing a layout-independent cryptographic stamp of a distributed data set from a data storage system, the system, computer program product, and computer-executable method comprising receiving a request for a cryptographic stamp of the distributed data set, creating a hash of each slice of the distributed data set, and using each hash from each slice of the distribute data set to create the cryptographic stamp of the distributed data set.

Abstract: A storage array uses both high endurance SSDs and low endurance SSDs for metadata paging. Wear cost values are calculated for each page of metadata in cache. The wear cost values are used to select pages for swapping out of the cache to the SSDs. The wear cost values may be calculated as a function of a first term that is indicative of whether the respective page of metadata will be written to high endurance or low endurance SSDs; a second term that is indicative of likelihood of data associated with the respective pages of metadata that will possibly be changed due to a write; and a third term that is indicative of age of the respective page of metadata in the cache since most recent use. The terms may be estimated and independently weighted. The portion of cache allocated for the metadata may be increased to avoid exceeding DWPD targets.

Abstract: Described embodiments provide systems and methods for operating a storage system. A portion of data currently on a source Logical Unit (LUN) of a disk array of a plurality of disk arrays is written to a destination LUN of a disk array of a plurality of disk arrays. The destination LUN of a disk array of the plurality of disk arrays to write the data to is established based on an efficiency indicator associated with the destination LUN of a disk array of the plurality of disk arrays. The writing at least a portion of data currently on a source LUN of a disk array of a plurality of disk arrays may be performed as part of a replication process or as part of a relocation process.

Abstract: An apparatus in one embodiment comprises at least one processing device comprising a processor coupled to a memory. The processing device is configured to capture metadata from a metadata area of a storage system for checkpointing, and to store the captured metadata as checkpointed metadata in a user data area of the storage system in association with mapping information relating portions of the checkpointed metadata to particular storage locations in the user data area. The processing device in some embodiments is implemented within the storage system, and may comprise a storage controller of the storage system. The mapping information illustratively comprises a plurality of page location arrays each having a plurality of entries specifying respective locations of checkpointed metadata pages in the user data area.

Abstract: A storage system comprising a processor in operable communication with a storage device is configured to have first, second, and third tables and first, second, and third respective metadata journals storing time-based information associated, respectively, with the first, second, and third tables. The first table maps logical addresses associated with LUNs associated with the storage device to respective hash digests computed based on data residing at respective logical addresses. The second table maps a range of hash digests to respective associated physical locations on the storage device. The third table maps disk block allocation on the storage device. An online system checkpoint is created, comprising information relating to the state of the storage system at a point in time, where the first, second, and third, metadata journals are destaged in order so as to destage information stored in the respective journal up to the point in time.

Abstract: An apparatus in one embodiment comprises a storage system having a plurality of storage devices and an associated storage controller. The storage devices are configured to provide a metadata area and a user data area, the user data area being separated into pages. The storage controller is configured to initiate generation of one or more checkpoints in the storage system. In conjunction with the generation of a given one of the one or more checkpoints in the storage system, the storage controller causes checkpointed metadata from the metadata area to be stored in the user data area. The storage controller further stores mapping information relating portions of the checkpointed metadata to respective pages of the user data area that store those portions of the checkpointed metadata. The mapping information illustratively comprises page location arrays and array location arrays, and in some embodiments further comprises one or more higher-level location arrays.

Abstract: A computer-implemented method is provided for a storage system comprising a processor in operable communication with a storage device that is configured into a plurality of respective regions of storage space. A first defragmentation process is performed on at least a first region of the plurality of respective regions of storage space to free a first portion of storage space in the a respective first region, where the freed first portion is reserved for user data and metadata associated with I/O requests to the storage system that are generated during a first operation running on the storage system. The storage system is configured to ensure that, during the first operation, user data and metadata are routed towards the first freed portion and kept separate from a set of system checkpoint information, associated with a first state of the storage system, stored in a protected portion of storage space.

Abstract: In a system in which a plurality of computing elements share a cache, each computing element owns a stripe of the cache. Each stripe contains cache objects that are accessible to all computing elements but managed only by the owning computing element. Each computing element maintains an LRU FIFO queue in local memory for the cache objects owned by that computing element. Each computing element also maintains a separate hash table in local memory for each other computing element. The hash tables indicate access to cache objects that are owned by those other computing elements. Each computing element updates its LRU FIFO queue when it accesses cache objects that it owns. The hash tables are periodically distributed by all computing elements via RDMA so that the LRU FIFO queues of all computing elements can be updated based on accesses to owned cache objects by other non-owner computing elements.

Abstract: In one aspect, a method includes migrating a first device on a first array to a second device on a second storage array. The migrating includes instructing an asset to scan the first array and the second array for the first device and the second device, reading a work buffer on the second array, sending a request to the work buffer for a chunk of the first device to copy to the second device, reading assigned chunk from the first device, copying the assigned chunk to the second device and notifying the work buffer that the copying of the assigned chunk was successful.

Abstract: Example embodiments relate to a method, a system, and a computer program product for load balancing for port selection. The method includes determining a processing load for each storage port in a plurality of storage ports having variable processing power and calculating a delay characteristic for each storage port of the plurality of storage ports according to its processing load. A command then may be sent to a selected storage port of the plurality of storage ports according to the delay characteristics and a policy.

Abstract: In one aspect, a method includes sending a conditional read request from a host to a storage array requesting data in a data block stored at the storage array. The conditional read request includes a first hash of data in the data block at the host. The method also includes determining a second hash of the data in the data block stored at the storage array, comparing the first hash and the second hash, sending a reply from the storage array to the host with the data in the data block stored at the storage array if the first hash and the second hash differ and sending a reply from the storage array to the host without the data in the data block stored at the storage array if the first hash and the second hash are the same.