Abstract: A method for data defragmentation is disclosed, including: selecting a file that is stored in a content-addressable storage system, the file including a plurality of blocks, and the storage system including a plurality of deduplication chunks; arranging the plurality of blocks in block groups, each block group including a set of consecutive blocks having a combined size that matches a deduplication chunk size of the storage system; aligning each block group with a different one of the plurality of deduplication chunks of the storage system, such that the blocks in each block group are stored in the storage system in an order that is based on the order in which the blocks are positioned within the file.

Abstract: A lock controller is connected to a primary lock server having a primary memory to implement a primary ring buffer lock and a secondary lock server having a secondary memory to implement a mirrored primary ring buffer lock. The lock controller sends a first lock request and first read request to the primary lock server and receives a first order. Further, the lock controller sends a second lock request and a second read request to the secondary lock server and receives a second order. Further, the lock controller compares the first and second order, and if the second order differs from the first order, then sends a reorder request to the secondary lock server indicating the first order, so that the execution order of lock requests can be maintained. The disclosed lock controller ensures low latency and efficiently handles a server failure.

Abstract: Provided is a storage controller for controlling a storage memory. The storage memory includes at least one first tier memory and at least one second tier memory. The first tier memory has a higher access speed than the second tier memory. The storage memory is being arranged to store memory blocks of data grouped in memory chunks associated with an index. A memory chunk indicates the size of a memory operation. The storage controller is configured to determine that a memory chunk is to be moved from the first tier memory to the second tier memory, and write the memory chunk to the second tier memory. The storage controller is configured to receive a read request for the memory chunk and in response thereto determine whether the memory chunk is current in the first tier memory, and if so, read the memory chunk from the first tier memory.

Abstract: A memory controller stores data comprising a plurality of data blocks in a first memory tier by data striping data on data drives of a first drive type. The memory controller determines a first parity for at least one data stripe of a plurality of data stripes, stores the first parity in one of data drives of the first drive type, determines a plurality of second parities for the at least one data stripe of the plurality of data stripes, and stores the second parities in a second memory tier.

Abstract: A block storage device, for data compression is configured to, in a first operating phase, if it is determined, by the block storage device, that a data block is to be written to a large block storage area of the block storage device, determine if the data block can be de-duplicated. If the data block cannot be de-duplicated, the block storage device stores the data block using large block compression.

Abstract: A storage system in one embodiment comprises a plurality of storage devices and a storage controller. The storage controller is configured to receive information indicating whether target data is one of a non-deduplicatable type of data and a deduplicatable type of data. Responsive to the information indicating that the target data is of the deduplicatable type, the storage controller is configured to utilize a content-based mapping generated based on a content of the target data to identify a corresponding physical address for storing the target data. Responsive to the information indicating that the target data is of the non-deduplicatable type, the storage controller is configured to utilize an address-based mapping generated based on a logical address associated with the target data to identify a corresponding physical address for storing the target data.

Abstract: Example embodiments of the present invention relate and a method and an apparatus for double hashing. The method including receiving a hash signature, including a short hash handle, for a data block. The method then includes determining a bucket with which the hash signature should be associated and associating the hash signature with the bucket.

Abstract: In an information processing system including a set of data storage devices for storing data blocks arranged in respective columns on each data storage device and rows across the set of data storage devices to form at least one data stripe, and a set of parity storage devices for storing parity blocks computed via one or more parity operations based on the data blocks of the at least one data stripe, at least one of the data storage devices includes a processing device configured to: receive from the information processing system an instruction to perform at least a portion of a parity operation; perform the portion of the parity operation; and send a result of the performed portion of the parity operation to the information processing system, wherein the result is useable by the information processing system for performing another portion of the parity operation.

Abstract: A storage system comprises a plurality of enclosures and a storage controller. Each enclosure comprises at least one processing device and a plurality of drives configured in accordance with a redundant array of independent disks (RAID) arrangement. The storage controller obtains data pages associated with an input-output request, provides the data pages to a processing device of a given enclosure, and issues a command to the processing device to perform at least one operation based at least in part on the data pages. The processing device of the given enclosure receives the data pages from the storage controller, generates compressed data pages based at least in part on the received data pages, stores one or more of the compressed data pages on the plurality of drives according to the RAID arrangement and returns information associated with the storage of the compressed data pages to the storage controller.

Abstract: An aspect includes reading a first page and a corresponding second page from a storage device. The first page specifies a metadata page stored in persistent storage and having logical addresses of metadata, and the second page associates logical block addresses (LBAs) with corresponding physical locations for the metadata. An aspect also includes reading data for a RAID stripe according to an associated physical offset in the second page, accessing a stripe counter from the second page, and comparing the stripe counter from the second page to a stripe counter held in memory. Upon determining the stripe counter from the second page is not the same, a third page is loaded, and a physical location of the data is read from the third page that provides a hash value of the data and corresponding physical location. The physical location of the data is accessed, and the second page is updated.

Abstract: A method for performing host based read cache for storage area networks (SANs) supporting Non-Volatile Memory Express over Fabrics (NVMeF) with validation. The method includes upon a first read operation from a client system of a storage array, receiving metadata with a read response regarding location of data subject to the read operation. The method also includes performing, by the client system, direct NVMeF reads from a namespace offset from the metadata. The direct NVMeF reads bypass a software stack of the storage array. The method further includes upon receiving a data block at the client system, using a signature from the metadata to perform validation on the data.

Abstract: An apparatus, in one example, comprises a storage system configured to perform one or more deduplication operations on encrypted datasets received for a plurality of tenants and store at least a portion of the encrypted datasets, the datasets having been encrypted for respective ones of the plurality of tenants using a common encryption key. The apparatus further comprises a cryptographic module associated with the storage system, the cryptographic module configured to, in response to a request to access an encrypted dataset stored by the storage system corresponding to a given one of the plurality of tenants, further encrypt the encrypted dataset using a tenant encryption key associated with the given one of the plurality of tenants. The storage system is further configured to send the further encrypted dataset to the given one of the plurality of tenants that requested access.

Abstract: In one aspect, data replication processes include setting a minimum snap set creation interval, a maximum snap set creation interval, and a recovery time threshold (RTT). An aspect further includes creating snap sets during a synchronous replication process based on the snap set creation intervals and the RTT. The snap set creation further includes monitoring an amount of data changes since the last snap set creation, monitoring throughput statistics between a source and target system, and estimating an amount of time to replicate the data changes to the target system based on the data changes and throughput statistics. Upon determining the time to replicate the data changes reaches the RTT and the time since the last snap set creation passes the minimum interval, the snap set creation includes creating a next snap set, storing the next snap set at the source, and replicating the next snap set at the target.

Abstract: A method is disclosed for storing data in a storage system, comprising: receiving a write request at the storage system; detecting whether the write request is associated with one of a deduplication tier or a non-deduplication tier of the storage system; when the write request is associated with the deduplication tier of the storage system, storing data associated with the write request in the first portion of the volume and updating a first mapping structure to map the first LBA to at least one first hash digest that is generated based on the data; and when the write request is associated with the non-deduplication tier of the storage system, storing data associated with the write request in the second portion of the volume and updating the first mapping structure to map the first LBA to a second hash digest that is generated based on a second LBA.

Abstract: A storage system comprises a plurality of enclosures and a storage controller. Each enclosure comprises at least one processing device and a plurality of drives configured in accordance with a redundant array of independent disks (RAID) arrangement. The storage controller obtains an input-output request to read a target data page and issues a command to the at least one processing device of a given enclosure. Responsive to receiving the command, the at least one processing device of the given enclosure obtains data pages associated with a given stripe from the plurality of drives of the given enclosure. The at least one processing device rebuilds the given stripe based at least in part on the obtained data pages, obtains the target data page from the rebuilt given stripe and provides the obtained target data page to the storage controller.

Abstract: In one aspect, performance optimization for data persistency in asynchronous replication setups includes creating at a source site of a data replication system, a snapshot (snapshot N) of input/output (IO) requests as part of a replication cycle, computing a delta of snapshot N and a previously created snapshot (snapshot N?1), and transmitting the delta to a target site of the data replication storage system. An aspect further includes storing, at the target site, snapshot N?1 during transmission of the delta, and caching the delta to a non-persistent storage device. Upon determining an occurrence of a loss event at the target site, a further aspect includes transmitting, by the target site, a request to retransmit the delta, retransmitting, by the source site, the delta to the target site, caching the delta to the non-persistent storage device until successful transmission of the delta, and committing the transmitted delta to permanent storage.

Abstract: Described embodiments provide systems and methods for operating a storage system wherein an online consistent system checkpoint is generated. The checkpoint contains a point in time image of a system and is used for providing recovery of the system to a known good state. In one embodiment the checkpoint includes volume configuration data, logical volume space, a plurality of layers of metadata, and physical data storage.

Abstract: An apparatus in one embodiment comprises at least one processing device comprising a processor coupled to a memory. The at least one processing device is configured to obtain a file and to determine a type of the file. A structure of the file may be determined based at least in part on the determined type of the file and at least one location in the file may be identified based at least in part on the determined structure. The file may be divided at the identified at least one location into a plurality of chunks and the plurality of chunks may be provided to a block deduplication module of a storage system where the block deduplication module is configured to perform a deduplication process based at least in part on the plurality of chunks.

Abstract: An apparatus comprises at least one processing device comprising a processor coupled to a memory. The processing device is configured to select a stripe column size for stripes of a data storage system, to determine a first compress block size for a first one of the stripes based on compressibility of data to be stored, to select a first prime number for computing parity blocks for the first stripe and a first number of sub-stripes for splitting stripe columns of the first stripe, to generate metadata specifying the first compress block size, the first prime number and the first number of sub-stripes for the first stripe, and to store data compressed using the first compress block size in the first stripe. The first prime number and first number of sub-stripes for the first stripe is different than a second prime number and second number of sub-stripes for a second stripe.

Abstract: An apparatus comprises a storage system having storage devices and an associated storage controller. In conjunction with initiation of a checkpoint, the storage controller sets a checkpoint started flag for the checkpoint, marks user data pages and metadata pages for write operations already entered in a write journal of the storage system as of the setting of the checkpoint started flag as checkpoint pages, and marks user data pages and metadata pages for new write operations entered in the write journal after the setting of the checkpoint started flag as non-checkpoint pages by altering information used to generate signatures for respective ones of the metadata pages. Metadata pages characterizing the same user data pages subject to write operations at different times thereby have different signatures depending on whether or not the checkpoint started flag was set when its corresponding write operation was entered in the write journal.