Abstract: A method is provided for use in computing system, comprising: instantiating a plurality of scheduler queues; instantiating a plurality of type-1 threads; assigning each of the plurality of type-1 threads to a respective scheduler queue; allocating a plurality of budgets to the plurality of scheduling of queues, each of the plurality of budgets being allocated to a different one of the plurality of scheduling queues; selecting one of the plurality of scheduling queues; retrieving a type-1 thread from the selected scheduling queue when one of a first condition or a second condition is satisfied, the first condition being satisfied when the respective budget of the queue is not exhausted, and the second condition being satisfied when another one of the plurality of scheduling queues are empty; and executing the retrieved type-1 thread.

Abstract: A method, computer program product, and computing system for receiving, via a first node of a plurality of nodes, data for storage in a storage system. The data may be written, via the first node, to one or more data pages of a log buffer of the first node. One or more page descriptors associated with the one or more data pages may be generated via the first node. The one or more page descriptors may be sent to a second node concurrently with the writing of the data to the one or more data pages of the log buffer of the first node.

Abstract: A method for use in a storage system, the method comprising: receiving, at a first storage processor in the plurality, an Input/Output (I/O) request that is associated with a storage object; identifying an entity associated with the I/O request and the storage object; detecting, by the first storage processor, whether the first storage processor is a current owner of the storage object; when the first storage processor is the current owner of the storage object, setting a lock on the entity, the lock being set by the first storage processor, the lock being set independently of any other storage processors in the storage system; when the first storage processor is not the current owner of the storage object, setting the lock in cooperation with the current owner of the storage object; and executing the I/O request based on the entity after the lock has been set.

Abstract: A method of performing deduplication is provided. The method includes (a) selecting, by applying a deterministic selection criterion, a sub-block of a block of data that contains multiple sub-blocks; (b) performing a lookup, into a deduplication table, of a digest generated by hashing the selected sub-block, the lookup matching an entry indexed by the digest in the deduplication table, the entry identifying a previously processed block; and (c) effecting storage of the block, including pointing to the previously processed block. An apparatus, system, and computer program product for performing a similar method are also provided.

Abstract: A computer-implemented method, according to one embodiment, includes: storing information in a specified system memory location, attaching an external process to the specified system memory location in response to experiencing a system halt event, sending the information stored in the specified system memory location to a memory location associated with the external process, restarting the system in a recovery mode, retrieving the information from the external process, and using the retrieved information to restore the system to a state the system was in when the system halt event occurred. Other systems, methods, and computer program products are described in additional embodiments.

Abstract: A method, computer program product, and computer system for obtaining, by a computing device, one or more pages from a log to complete a write transaction. Parity of a line in a multi-line physical layer block may be calculated. The one or more pages may be written to the line in the multi-line physical layer block. The parity to the line may be written in the multi-line physical layer block. A bitmap associated with the physical layer block may be updated based upon, at least in part, writing the one or more pages and the parity to the line in the multi-line physical layer block.

Abstract: A controller-implemented method, according to one embodiment, includes: receiving, by a first controller, data. Metadata associated with the data is stored, by the first controller, in a specified system memory location. Second metadata received from a second controller is also stored, by the first controller, in the specified system memory location, thereby creating combined metadata. In response to the second controller entering a failed state: snapshots of the combined metadata are stored, by the first controller, to resilient storage at a predefined interval. Moreover, additional data continues to be received by the first controller. Metadata associated with the additional data is stored, by the first controller, in the specified system memory location, while changes to the metadata which occur between the snapshots of the combined metadata are also stored by the first controller. According to some approaches, the changes to the metadata are stored in a log structured array.

Abstract: A method, computer program product, and computing system for receiving, via a first node of a plurality of nodes, data for storage in a storage system. The data may be written, via the first node, to one or more data pages of a log buffer of the first node. One or more page descriptors associated with the one or more data pages may be generated via the first node. The one or more page descriptors may be sent to a second node concurrently with the writing of the data to the one or more data pages of the log buffer of the first node.

Abstract: Techniques for performing cache management includes partitioning entries of a hash table into buckets, wherein each of the buckets includes a portion of the entries of the hash table, configuring a cache, wherein the configuring includes allocating a section of the cache for exclusive use by each bucket, and performing first processing that stores a data block in the cache. The first processing includes determining a hash value for a data block, selecting, in accordance with the hash value, a first bucket of the plurality of buckets, wherein a first section of the cache is used exclusively for storing cached data blocks of the first bucket, storing metadata used in connection with caching the data block in a first entry of the first bucket, and storing the data block in a first cache location of the first section of the cache.

Abstract: A method, computer program product, and computer system for receiving, by a computing device, a Write-Same operation from a host for a range of logical block addresses of a destination. Data may be recorded in a buffer to indicate that the Write-Same operation is complete prior to completing the Write-Same operation. An acknowledgment may be sent to the host that the Write-Same operation is complete prior to flushing to a final destination. The Write-Same operation for the logical block addresses of the destination may be performed after sending the acknowledgment to the host that the Write-Same operation is complete.

Abstract: A method, computer program product, and computing system for receiving data for storage in a storage system. The data may be written to a head entry of a log buffer, wherein the log buffer includes a plurality of data entries for flushing to the storage system. At least a portion of the plurality of data entries of the log buffer may be flushed, via a plurality of threads, to the storage system based upon, at least in part, a tail entry of the log buffer. A queue of committed data entries may be updated, via each thread of the plurality of threads, with one or more data entries of the log buffer flushed to the storage system by each thread. A new tail entry of the log buffer may be determined, via a thread of the plurality of threads, based upon, at least in part, the queue of committed data entries.

Abstract: A method comprising: storing, in a memory, a mapping tree that is implemented by using an array of mapping pages, the mapping tree having a depth of D, wherein D is an integer greater than or equal to 0; receiving a write request that is associated with a first type-1 address; storing, in a storage device, data associated with the write request, the data associated with the write request being stored in the storage device based on a first type-2 address; generating a map entry that maps the first type-1 address to the first type-2 address; calculating a first hash digest of the first type-1 address; and storing the map entry in a first mapping page.

Abstract: A method is provided comprising: initializing a first thread based on a first work request; and executing the first thread to complete the first work request, wherein the first thread is initialized and executed by using an Application Programming Interface (API) that is arranged to emulate a work request queue by initializing a different respective thread for each work request that is submitted to the API for addition to the work request queue.

Abstract: Techniques for providing hiccup-less failback and journal recovery. The techniques include determining a head position of a data log of a primary node and placing a marker at a corresponding position in a data log of a secondary node. In response to an IO request, the head of the primary log is moved to allocate space for writing a data entry, and the secondary node is directed to move a head of the secondary log to allocate space for writing the data entry. In response to a data entry being flushed from the primary node, a tail of the primary log is moved to reclaim allocated space for a previous data entry, and the secondary node is directed to move a tail of the secondary log to reclaim previously allocated space. In response to a tail of the secondary log coinciding with the marker position, content of the logs is synchronized.

Abstract: Techniques for providing an efficient virtualization layer structure in a data storage system. The techniques include implementing multiple layers of indirection for accessing host data in the data storage system, including a mapping layer, a virtualization layer, and a physical layer. The virtualization layer includes virtual layer blocks (VLBs), each VLB including virtual pointers. Each virtual pointer is pointed to by one or more leaf pointers in the mapping layer, and points to a data block in the physical layer. The techniques include generating, for each virtual pointer in the VLB, reference count metadata to keep track of the number of leaf pointers pointing to the virtual pointer, and maintaining, in a metadata page, the reference count metadata for the virtual pointers in a three (3)-way mirror. The techniques include maintaining each VLB of the virtualization layer in a RAID stripe across multiple physical drives in the data storage system.

Abstract: A technique for performing deduplication traverses a deduplication database and assigns digest values in the database to buckets, where each bucket covers a respective range of digest values. To deduplicate a particular candidate block, the technique generates a digest from the candidate block and searches for the computed digest in a subset of the buckets, where the subset is selected based on the computed digest. If a target block providing an exact match or a suitably close partial match is found in the subset of buckets, the technique effects storage of the candidate block at least in part by providing a reference to the target block.

Abstract: Techniques for optimizing write IO bandwidth and latency in an active-active clustered system using storage object affinity to a single node. The active-active clustered system can include at least a primary storage node and a secondary storage node that maintain their own journals. The respective journals are directly accessible to both storage nodes. The journals are synchronized for each page or entity of a storage object when a storage IO request is issued to a storage node to which the storage object does not have affinity. Such synchronization is performed in the framework of acquiring a lock on the entity of the storage object during internode communications. To facilitate recovery from a disaster, data loss, and/or data corruption, transaction IDs associated with storage IO operations are employed to facilitate identification of the most up-to-date reference or description information for a given data or metadata page of a storage object.

Abstract: An aspect of implementing globally optimized partial deduplication of storage objects includes gathering pages that share a common feature, dividing the pages into groups based on commonality with corresponding representative pages, where each is assigned as a representative dedupe page for the corresponding groups. For each group in the groups of pages, an aspect also includes writing the pages to a corresponding physical container.

Abstract: A technique for validating metadata includes creating log entries for virtualization structures pointed to by mapping pointers in a mapping tree and processing the log entries in multiple passes. A current pass validates a current level of redirection and creates new log entries to be processed during a next pass. The new log entries represent a next level of redirection, and as many next passes are processed in sequence as there are next levels of redirection.

Abstract: Techniques for providing hiccup-less failback and journal recovery. The techniques include determining a head position of a data log of a primary node and placing a marker at a corresponding position in a data log of a secondary node. In response to an IO request, the head of the primary log is moved to allocate space for writing a data entry, and the secondary node is directed to move a head of the secondary log to allocate space for writing the data entry. In response to a data entry being flushed from the primary node, a tail of the primary log is moved to reclaim allocated space for a previous data entry, and the secondary node is directed to move a tail of the secondary log to reclaim previously allocated space. In response to a tail of the secondary log coinciding with the marker position, content of the logs is synchronized.